Re: [SR-Users] Possible memory leak dealing with presence in kamailio
The statistics of kamailio pkg and shm show rather steady values across the period of time. PKG is like less than 4MB for each process always and SHM stays free at 426MB. So obviously there is no leak in pkg and shm. Check the details at the next link for OS memory reports: - http://www.kamailio.org/wiki/tutorials/troubleshooting/memory#os_memory_reports It can be unclaimed free cache what you can see as used memory. Cheers, Daniel On 11/02/15 04:28, Nuno Reis wrote: Just to be clear. The memory is in Kilobytes (kB). -- *Nuno Miguel Reis*| *Unified Communication**Systems* M. +351 913907481 | nr...@wavecom.pt mailto:nr...@wavecom.pt WAVECOM-Soluções Rádio, S.A. Cacia Park | Rua do Progresso, Lote 15 3800-639 AVEIRO | Portugal T. +351 309 700 225 | F. +351 234 919 191 *GPS http://maps.google.com/maps/ms?msa=0msid=202333747613191340808.0004b4b227a6144f0df88 | www.wavecom.pt http://www.wavecom.pt/**http://www.wavecom.pt/* Description: Description: WavecomSignature http://www.wavecom.pt/pt/wavecom/premios.php Publicity http://www.wavecom.pt/pt/mail_eventos.php On Wed, Feb 11, 2015 at 3:26 AM, Nuno Reis nr...@wavecom.pt mailto:nr...@wavecom.pt wrote: Hi Daniel. Thank you for your suggestion and feedback on this. I've tried that already and here's what I've found after 15h on a running kamailio: All records in pua and presentity DB tables are gone by the end of the day, so the expire time seems to be working. I still see system memory growing and not being released again. You can find attached (tar.gz) various dumps of pkgstats and shmem usage. The number after the 'underscore' in file names corresponds to system memory allocation in kamailio at the various timestamps. The earlier timestamp corresponds to minute 1. So I started with 638632kb system mem allocation and I'm now with 1086548kb being used after 15 hours. I'll continue to investigate the issue but if you have any other suggestions on how to tackle this, I'm of course available to test those. Looking forward to hear from you. Best Regards, -- *Nuno Miguel Reis*| *Unified Communication**Systems* M. +351 913907481 | nr...@wavecom.pt mailto:nr...@wavecom.pt WAVECOM-Soluções Rádio, S.A. Cacia Park | Rua do Progresso, Lote 15 3800-639 AVEIRO | Portugal T. +351 309 700 225 | F. +351 234 919 191 *GPS http://maps.google.com/maps/ms?msa=0msid=202333747613191340808.0004b4b227a6144f0df88 | www.wavecom.pt http://www.wavecom.pt/**http://www.wavecom.pt/* Description: Description: WavecomSignature http://www.wavecom.pt/pt/wavecom/premios.php Publicity http://www.wavecom.pt/pt/mail_eventos.php On Mon, Feb 9, 2015 at 5:05 PM, Daniel-Constantin Mierla mico...@gmail.com mailto:mico...@gmail.com wrote: Hello, can you check the expires column in presentity table. The issue might be accumulation of too many dialog-info documents, due to large expires interval, taken from the default lifetime of the dialog. You can change that with: - http://kamailio.org/docs/modules/4.2.x/modules/pua_dialoginfo.html#idp2576952 Cheers, Daniel On 30/01/15 16:43, Nuno Reis wrote: Hi Daniel. Thanks for answering me back. I'll follow the exact procedures from here: http://www.kamailio.org/wiki/tutorials/troubleshooting/memory and will let you know about my exact finding soon. Cheers, -- *Nuno Miguel Reis*| *Unified Communication**Systems* M. +351 913907481 | nr...@wavecom.pt mailto:nr...@wavecom.pt WAVECOM-Soluções Rádio, S.A. Cacia Park | Rua do Progresso, Lote 15 3800-639 AVEIRO | Portugal T. +351 309 700 225 | F. +351 234 919 191 *GPS http://maps.google.com/maps/ms?msa=0msid=202333747613191340808.0004b4b227a6144f0df88 | www.wavecom.pt http://www.wavecom.pt/**http://www.wavecom.pt/* Description: Description: WavecomSignature http://www.wavecom.pt/pt/wavecom/premios.php Publicity http://www.wavecom.pt/pt/mail_eventos.php On Fri, Jan 30, 2015 at 5:23 AM, Daniel-Constantin Mierla mico...@gmail.com mailto:mico...@gmail.com wrote: Hello, which memory is increasing? shared or private memory? or is system memory? Cheers, Daniel On Fri, Jan 30, 2015 at 4:24 AM, Nuno Reis nr...@wavecom.pt mailto:nr...@wavecom.pt wrote: Hi Juha and all. I understand that and that is what the RFC says. It seems pua module does that right. Although something is clearly not right in my production environment because kamailio memory
Re: [SR-Users] Possible memory leak dealing with presence in kamailio
Just to be clear. The memory is in Kilobytes (kB). -- *Nuno Miguel Reis* | *Unified Communication** Systems* M. +351 913907481 | nr...@wavecom.pt WAVECOM-Soluções Rádio, S.A. Cacia Park | Rua do Progresso, Lote 15 3800-639 AVEIRO | Portugal T. +351 309 700 225 | F. +351 234 919 191 *GPS http://maps.google.com/maps/ms?msa=0msid=202333747613191340808.0004b4b227a6144f0df88 | www.wavecom.pt http://www.wavecom.pt/** http://www.wavecom.pt/* [image: Description: Description: WavecomSignature] http://www.wavecom.pt/pt/wavecom/premios.php [image: Publicity] http://www.wavecom.pt/pt/mail_eventos.php On Wed, Feb 11, 2015 at 3:26 AM, Nuno Reis nr...@wavecom.pt wrote: Hi Daniel. Thank you for your suggestion and feedback on this. I've tried that already and here's what I've found after 15h on a running kamailio: All records in pua and presentity DB tables are gone by the end of the day, so the expire time seems to be working. I still see system memory growing and not being released again. You can find attached (tar.gz) various dumps of pkgstats and shmem usage. The number after the 'underscore' in file names corresponds to system memory allocation in kamailio at the various timestamps. The earlier timestamp corresponds to minute 1. So I started with 638632kb system mem allocation and I'm now with 1086548kb being used after 15 hours. I'll continue to investigate the issue but if you have any other suggestions on how to tackle this, I'm of course available to test those. Looking forward to hear from you. Best Regards, -- *Nuno Miguel Reis* | *Unified Communication** Systems* M. +351 913907481 | nr...@wavecom.pt WAVECOM-Soluções Rádio, S.A. Cacia Park | Rua do Progresso, Lote 15 3800-639 AVEIRO | Portugal T. +351 309 700 225 | F. +351 234 919 191 *GPS http://maps.google.com/maps/ms?msa=0msid=202333747613191340808.0004b4b227a6144f0df88 | www.wavecom.pt http://www.wavecom.pt/** http://www.wavecom.pt/* [image: Description: Description: WavecomSignature] http://www.wavecom.pt/pt/wavecom/premios.php [image: Publicity] http://www.wavecom.pt/pt/mail_eventos.php On Mon, Feb 9, 2015 at 5:05 PM, Daniel-Constantin Mierla mico...@gmail.com wrote: Hello, can you check the expires column in presentity table. The issue might be accumulation of too many dialog-info documents, due to large expires interval, taken from the default lifetime of the dialog. You can change that with: - http://kamailio.org/docs/modules/4.2.x/modules/pua_dialoginfo.html#idp2576952 Cheers, Daniel On 30/01/15 16:43, Nuno Reis wrote: Hi Daniel. Thanks for answering me back. I'll follow the exact procedures from here: http://www.kamailio.org/wiki/tutorials/troubleshooting/memory and will let you know about my exact finding soon. Cheers, -- *Nuno Miguel Reis* | *Unified Communication** Systems* M. +351 913907481 | nr...@wavecom.pt WAVECOM-Soluções Rádio, S.A. Cacia Park | Rua do Progresso, Lote 15 3800-639 AVEIRO | Portugal T. +351 309 700 225 | F. +351 234 919 191 *GPS http://maps.google.com/maps/ms?msa=0msid=202333747613191340808.0004b4b227a6144f0df88 | www.wavecom.pt http://www.wavecom.pt/** http://www.wavecom.pt/* [image: Description: Description: WavecomSignature] http://www.wavecom.pt/pt/wavecom/premios.php [image: Publicity] http://www.wavecom.pt/pt/mail_eventos.php On Fri, Jan 30, 2015 at 5:23 AM, Daniel-Constantin Mierla mico...@gmail.com wrote: Hello, which memory is increasing? shared or private memory? or is system memory? Cheers, Daniel On Fri, Jan 30, 2015 at 4:24 AM, Nuno Reis nr...@wavecom.pt wrote: Hi Juha and all. I understand that and that is what the RFC says. It seems pua module does that right. Although something is clearly not right in my production environment because kamailio memory consumption still grows pretty fast. Kamailio memory usage starts in ~500MB and after ~24H kamailio is using ~3GB. If I disable kamailio from listening on the localhost(127.0.0.1) where pua is generating the SIP Publishes kamailio just keeps around the ~500MB all the time. This is a small production environment with 70 extensions with Yealink phones. Any ideas on how to chase down this memory leak? Should I open a git issue for this one? -- *Nuno Miguel Reis* | *Unified Communication** Systems* M. +351 913907481 | nr...@wavecom.pt WAVECOM-Soluções Rádio, S.A. Cacia Park | Rua do Progresso, Lote 15 3800-639 AVEIRO | Portugal T. +351 309 700 225 | F. +351 234 919 191 *GPS http://maps.google.com/maps/ms?msa=0msid=202333747613191340808.0004b4b227a6144f0df88 | www.wavecom.pt http://www.wavecom.pt/** http://www.wavecom.pt/* [image: Description: Description: WavecomSignature] http://www.wavecom.pt/pt/wavecom/premios.php [image: Publicity] http://www.wavecom.pt/pt/mail_eventos.php On Wed, Jan 21, 2015 at 8:45 PM, Juha Heinanen j...@tutpro.com wrote: Nuno Reis writes: Here my publisher is Kamailio itself. Can
Re: [SR-Users] Possible memory leak dealing with presence in kamailio
Hi Daniel. Thank you for your suggestion and feedback on this. I've tried that already and here's what I've found after 15h on a running kamailio: All records in pua and presentity DB tables are gone by the end of the day, so the expire time seems to be working. I still see system memory growing and not being released again. You can find attached (tar.gz) various dumps of pkgstats and shmem usage. The number after the 'underscore' in file names corresponds to system memory allocation in kamailio at the various timestamps. The earlier timestamp corresponds to minute 1. So I started with 638632kb system mem allocation and I'm now with 1086548kb being used after 15 hours. I'll continue to investigate the issue but if you have any other suggestions on how to tackle this, I'm of course available to test those. Looking forward to hear from you. Best Regards, -- *Nuno Miguel Reis* | *Unified Communication** Systems* M. +351 913907481 | nr...@wavecom.pt WAVECOM-Soluções Rádio, S.A. Cacia Park | Rua do Progresso, Lote 15 3800-639 AVEIRO | Portugal T. +351 309 700 225 | F. +351 234 919 191 *GPS http://maps.google.com/maps/ms?msa=0msid=202333747613191340808.0004b4b227a6144f0df88 | www.wavecom.pt http://www.wavecom.pt/** http://www.wavecom.pt/* [image: Description: Description: WavecomSignature] http://www.wavecom.pt/pt/wavecom/premios.php [image: Publicity] http://www.wavecom.pt/pt/mail_eventos.php On Mon, Feb 9, 2015 at 5:05 PM, Daniel-Constantin Mierla mico...@gmail.com wrote: Hello, can you check the expires column in presentity table. The issue might be accumulation of too many dialog-info documents, due to large expires interval, taken from the default lifetime of the dialog. You can change that with: - http://kamailio.org/docs/modules/4.2.x/modules/pua_dialoginfo.html#idp2576952 Cheers, Daniel On 30/01/15 16:43, Nuno Reis wrote: Hi Daniel. Thanks for answering me back. I'll follow the exact procedures from here: http://www.kamailio.org/wiki/tutorials/troubleshooting/memory and will let you know about my exact finding soon. Cheers, -- *Nuno Miguel Reis* | *Unified Communication** Systems* M. +351 913907481 | nr...@wavecom.pt WAVECOM-Soluções Rádio, S.A. Cacia Park | Rua do Progresso, Lote 15 3800-639 AVEIRO | Portugal T. +351 309 700 225 | F. +351 234 919 191 *GPS http://maps.google.com/maps/ms?msa=0msid=202333747613191340808.0004b4b227a6144f0df88 | www.wavecom.pt http://www.wavecom.pt/** http://www.wavecom.pt/* [image: Description: Description: WavecomSignature] http://www.wavecom.pt/pt/wavecom/premios.php [image: Publicity] http://www.wavecom.pt/pt/mail_eventos.php On Fri, Jan 30, 2015 at 5:23 AM, Daniel-Constantin Mierla mico...@gmail.com wrote: Hello, which memory is increasing? shared or private memory? or is system memory? Cheers, Daniel On Fri, Jan 30, 2015 at 4:24 AM, Nuno Reis nr...@wavecom.pt wrote: Hi Juha and all. I understand that and that is what the RFC says. It seems pua module does that right. Although something is clearly not right in my production environment because kamailio memory consumption still grows pretty fast. Kamailio memory usage starts in ~500MB and after ~24H kamailio is using ~3GB. If I disable kamailio from listening on the localhost(127.0.0.1) where pua is generating the SIP Publishes kamailio just keeps around the ~500MB all the time. This is a small production environment with 70 extensions with Yealink phones. Any ideas on how to chase down this memory leak? Should I open a git issue for this one? -- *Nuno Miguel Reis* | *Unified Communication** Systems* M. +351 913907481 | nr...@wavecom.pt WAVECOM-Soluções Rádio, S.A. Cacia Park | Rua do Progresso, Lote 15 3800-639 AVEIRO | Portugal T. +351 309 700 225 | F. +351 234 919 191 *GPS http://maps.google.com/maps/ms?msa=0msid=202333747613191340808.0004b4b227a6144f0df88 | www.wavecom.pt http://www.wavecom.pt/** http://www.wavecom.pt/* [image: Description: Description: WavecomSignature] http://www.wavecom.pt/pt/wavecom/premios.php [image: Publicity] http://www.wavecom.pt/pt/mail_eventos.php On Wed, Jan 21, 2015 at 8:45 PM, Juha Heinanen j...@tutpro.com wrote: Nuno Reis writes: Here my publisher is Kamailio itself. Can someone elaborate a bit more on this issue and maybe we can get to bottom of it? when your application issues initial publish request, it does so without SIP-If-Match header. 200 ok from presence server then contains an etag in SIP-ETag header. when your application refreshes the publish, it must place this etag in SIP-If-Match header to prevent presence server from creating a new publication. for subscribes, your application must place in re-subscribe the same event header id param as the previous one had in order for the presence server to know that subscribe was not a new subscription. -- juha -- Daniel-Constantin Mierla - http://www.asipto.com
Re: [SR-Users] Possible memory leak dealing with presence in kamailio
Hello, can you check the expires column in presentity table. The issue might be accumulation of too many dialog-info documents, due to large expires interval, taken from the default lifetime of the dialog. You can change that with: - http://kamailio.org/docs/modules/4.2.x/modules/pua_dialoginfo.html#idp2576952 Cheers, Daniel On 30/01/15 16:43, Nuno Reis wrote: Hi Daniel. Thanks for answering me back. I'll follow the exact procedures from here: http://www.kamailio.org/wiki/tutorials/troubleshooting/memory and will let you know about my exact finding soon. Cheers, -- *Nuno Miguel Reis*| *Unified Communication**Systems* M. +351 913907481 | nr...@wavecom.pt mailto:nr...@wavecom.pt WAVECOM-Soluções Rádio, S.A. Cacia Park | Rua do Progresso, Lote 15 3800-639 AVEIRO | Portugal T. +351 309 700 225 | F. +351 234 919 191 *GPS http://maps.google.com/maps/ms?msa=0msid=202333747613191340808.0004b4b227a6144f0df88 | www.wavecom.pt http://www.wavecom.pt/**http://www.wavecom.pt/* Description: Description: WavecomSignature http://www.wavecom.pt/pt/wavecom/premios.php Publicity http://www.wavecom.pt/pt/mail_eventos.php On Fri, Jan 30, 2015 at 5:23 AM, Daniel-Constantin Mierla mico...@gmail.com mailto:mico...@gmail.com wrote: Hello, which memory is increasing? shared or private memory? or is system memory? Cheers, Daniel On Fri, Jan 30, 2015 at 4:24 AM, Nuno Reis nr...@wavecom.pt mailto:nr...@wavecom.pt wrote: Hi Juha and all. I understand that and that is what the RFC says. It seems pua module does that right. Although something is clearly not right in my production environment because kamailio memory consumption still grows pretty fast. Kamailio memory usage starts in ~500MB and after ~24H kamailio is using ~3GB. If I disable kamailio from listening on the localhost(127.0.0.1) where pua is generating the SIP Publishes kamailio just keeps around the ~500MB all the time. This is a small production environment with 70 extensions with Yealink phones. Any ideas on how to chase down this memory leak? Should I open a git issue for this one? -- *Nuno Miguel Reis*| *Unified Communication**Systems* M. +351 913907481 | nr...@wavecom.pt mailto:nr...@wavecom.pt WAVECOM-Soluções Rádio, S.A. Cacia Park | Rua do Progresso, Lote 15 3800-639 AVEIRO | Portugal T. +351 309 700 225 | F. +351 234 919 191 *GPS http://maps.google.com/maps/ms?msa=0msid=202333747613191340808.0004b4b227a6144f0df88 | www.wavecom.pt http://www.wavecom.pt/**http://www.wavecom.pt/* Description: Description: WavecomSignature http://www.wavecom.pt/pt/wavecom/premios.php Publicity http://www.wavecom.pt/pt/mail_eventos.php On Wed, Jan 21, 2015 at 8:45 PM, Juha Heinanen j...@tutpro.com mailto:j...@tutpro.com wrote: Nuno Reis writes: Here my publisher is Kamailio itself. Can someone elaborate a bit more on this issue and maybe we can get to bottom of it? when your application issues initial publish request, it does so without SIP-If-Match header. 200 ok from presence server then contains an etag in SIP-ETag header. when your application refreshes the publish, it must place this etag in SIP-If-Match header to prevent presence server from creating a new publication. for subscribes, your application must place in re-subscribe the same event header id param as the previous one had in order for the presence server to know that subscribe was not a new subscription. -- juha -- Daniel-Constantin Mierla - http://www.asipto.com http://twitter.com/#!/miconda http://twitter.com/#%21/miconda - http://www.linkedin.com/in/micond http://www.linkedin.com/in/miconda -- Daniel-Constantin Mierla http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda Kamailio World Conference, May 27-29, 2015 Berlin, Germany - http://www.kamailioworld.com ___ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
Re: [SR-Users] Possible memory leak dealing with presence in kamailio
Hi Daniel. Thanks for answering me back. I'll follow the exact procedures from here: http://www.kamailio.org/wiki/tutorials/troubleshooting/memory and will let you know about my exact finding soon. Cheers, -- *Nuno Miguel Reis* | *Unified Communication** Systems* M. +351 913907481 | nr...@wavecom.pt WAVECOM-Soluções Rádio, S.A. Cacia Park | Rua do Progresso, Lote 15 3800-639 AVEIRO | Portugal T. +351 309 700 225 | F. +351 234 919 191 *GPS http://maps.google.com/maps/ms?msa=0msid=202333747613191340808.0004b4b227a6144f0df88 | www.wavecom.pt http://www.wavecom.pt/** http://www.wavecom.pt/* [image: Description: Description: WavecomSignature] http://www.wavecom.pt/pt/wavecom/premios.php [image: Publicity] http://www.wavecom.pt/pt/mail_eventos.php On Fri, Jan 30, 2015 at 5:23 AM, Daniel-Constantin Mierla mico...@gmail.com wrote: Hello, which memory is increasing? shared or private memory? or is system memory? Cheers, Daniel On Fri, Jan 30, 2015 at 4:24 AM, Nuno Reis nr...@wavecom.pt wrote: Hi Juha and all. I understand that and that is what the RFC says. It seems pua module does that right. Although something is clearly not right in my production environment because kamailio memory consumption still grows pretty fast. Kamailio memory usage starts in ~500MB and after ~24H kamailio is using ~3GB. If I disable kamailio from listening on the localhost(127.0.0.1) where pua is generating the SIP Publishes kamailio just keeps around the ~500MB all the time. This is a small production environment with 70 extensions with Yealink phones. Any ideas on how to chase down this memory leak? Should I open a git issue for this one? -- *Nuno Miguel Reis* | *Unified Communication** Systems* M. +351 913907481 | nr...@wavecom.pt WAVECOM-Soluções Rádio, S.A. Cacia Park | Rua do Progresso, Lote 15 3800-639 AVEIRO | Portugal T. +351 309 700 225 | F. +351 234 919 191 *GPS http://maps.google.com/maps/ms?msa=0msid=202333747613191340808.0004b4b227a6144f0df88 | www.wavecom.pt http://www.wavecom.pt/** http://www.wavecom.pt/* [image: Description: Description: WavecomSignature] http://www.wavecom.pt/pt/wavecom/premios.php [image: Publicity] http://www.wavecom.pt/pt/mail_eventos.php On Wed, Jan 21, 2015 at 8:45 PM, Juha Heinanen j...@tutpro.com wrote: Nuno Reis writes: Here my publisher is Kamailio itself. Can someone elaborate a bit more on this issue and maybe we can get to bottom of it? when your application issues initial publish request, it does so without SIP-If-Match header. 200 ok from presence server then contains an etag in SIP-ETag header. when your application refreshes the publish, it must place this etag in SIP-If-Match header to prevent presence server from creating a new publication. for subscribes, your application must place in re-subscribe the same event header id param as the previous one had in order for the presence server to know that subscribe was not a new subscription. -- juha -- Daniel-Constantin Mierla - http://www.asipto.com http://twitter.com/#!/miconda - http://www.linkedin.com/in/micond http://www.linkedin.com/in/miconda ___ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
Re: [SR-Users] Possible memory leak dealing with presence in kamailio
Hi Juha and all. I understand that and that is what the RFC says. It seems pua module does that right. Although something is clearly not right in my production environment because kamailio memory consumption still grows pretty fast. Kamailio memory usage starts in ~500MB and after ~24H kamailio is using ~3GB. If I disable kamailio from listening on the localhost(127.0.0.1) where pua is generating the SIP Publishes kamailio just keeps around the ~500MB all the time. This is a small production environment with 70 extensions with Yealink phones. Any ideas on how to chase down this memory leak? Should I open a git issue for this one? -- *Nuno Miguel Reis* | *Unified Communication** Systems* M. +351 913907481 | nr...@wavecom.pt WAVECOM-Soluções Rádio, S.A. Cacia Park | Rua do Progresso, Lote 15 3800-639 AVEIRO | Portugal T. +351 309 700 225 | F. +351 234 919 191 *GPS http://maps.google.com/maps/ms?msa=0msid=202333747613191340808.0004b4b227a6144f0df88 | www.wavecom.pt http://www.wavecom.pt/** http://www.wavecom.pt/* [image: Description: Description: WavecomSignature] http://www.wavecom.pt/pt/wavecom/premios.php [image: Publicity] http://www.wavecom.pt/pt/mail_eventos.php On Wed, Jan 21, 2015 at 8:45 PM, Juha Heinanen j...@tutpro.com wrote: Nuno Reis writes: Here my publisher is Kamailio itself. Can someone elaborate a bit more on this issue and maybe we can get to bottom of it? when your application issues initial publish request, it does so without SIP-If-Match header. 200 ok from presence server then contains an etag in SIP-ETag header. when your application refreshes the publish, it must place this etag in SIP-If-Match header to prevent presence server from creating a new publication. for subscribes, your application must place in re-subscribe the same event header id param as the previous one had in order for the presence server to know that subscribe was not a new subscription. -- juha ___ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
Re: [SR-Users] Possible memory leak dealing with presence in kamailio
Hello, which memory is increasing? shared or private memory? or is system memory? Cheers, Daniel On Fri, Jan 30, 2015 at 4:24 AM, Nuno Reis nr...@wavecom.pt wrote: Hi Juha and all. I understand that and that is what the RFC says. It seems pua module does that right. Although something is clearly not right in my production environment because kamailio memory consumption still grows pretty fast. Kamailio memory usage starts in ~500MB and after ~24H kamailio is using ~3GB. If I disable kamailio from listening on the localhost(127.0.0.1) where pua is generating the SIP Publishes kamailio just keeps around the ~500MB all the time. This is a small production environment with 70 extensions with Yealink phones. Any ideas on how to chase down this memory leak? Should I open a git issue for this one? -- *Nuno Miguel Reis* | *Unified Communication** Systems* M. +351 913907481 | nr...@wavecom.pt WAVECOM-Soluções Rádio, S.A. Cacia Park | Rua do Progresso, Lote 15 3800-639 AVEIRO | Portugal T. +351 309 700 225 | F. +351 234 919 191 *GPS http://maps.google.com/maps/ms?msa=0msid=202333747613191340808.0004b4b227a6144f0df88 | www.wavecom.pt http://www.wavecom.pt/** http://www.wavecom.pt/* [image: Description: Description: WavecomSignature] http://www.wavecom.pt/pt/wavecom/premios.php [image: Publicity] http://www.wavecom.pt/pt/mail_eventos.php On Wed, Jan 21, 2015 at 8:45 PM, Juha Heinanen j...@tutpro.com wrote: Nuno Reis writes: Here my publisher is Kamailio itself. Can someone elaborate a bit more on this issue and maybe we can get to bottom of it? when your application issues initial publish request, it does so without SIP-If-Match header. 200 ok from presence server then contains an etag in SIP-ETag header. when your application refreshes the publish, it must place this etag in SIP-If-Match header to prevent presence server from creating a new publication. for subscribes, your application must place in re-subscribe the same event header id param as the previous one had in order for the presence server to know that subscribe was not a new subscription. -- juha -- Daniel-Constantin Mierla - http://www.asipto.com http://twitter.com/#!/miconda - http://www.linkedin.com/in/micond http://www.linkedin.com/in/miconda ___ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
Re: [SR-Users] Possible memory leak dealing with presence in kamailio
Hello all. Just want to tell you that the problem remains even after the patch Daniel add to the latest 4.2. I've sent another email yesterday (partially quoted bellow) to the mailing list which was already answered by Juha Heinanen in cc regarding an odd behavior (seems odd to me) i'm seeing. I've been testing presence module for a while and I do notice that presentity table grows endlessly over time. Each time a call is made a new record is added in presentity with a different etag. In documentation, namely developers guide we can read this: int etag_not_new; /* * 0 - the standard mechanism (allocating new etag for each Publish) * 1 - allocating an etag only for an initial Publish */ How can I tell the presence module in kamailio config to work using the second form of the above? I have a small production environment with ~70 extensions and in less than 24H (~20H), the presentity table grew from 0 to about ~2000 records, pua table grew to about ~850 records and kamailio memory usage grew from 600MB to 2GB on a 4G Linux server. Yesterday Juha said and I'll quote: presence module does not do anything with calls. it handles publish and subscriber requests and generates notifies. when publish is handled, it is the job of the publisher to place correct etag in subsequent refresh publish requests. -- juha Here my publisher is Kamailio itself. Can someone elaborate a bit more on this issue and maybe we can get to bottom of it? Thanks. Cheers, -- *Nuno Miguel Reis* | *Unified Communication** Systems* M. +351 913907481 | nr...@wavecom.pt WAVECOM-Soluções Rádio, S.A. Cacia Park | Rua do Progresso, Lote 15 3800-639 AVEIRO | Portugal T. +351 309 700 225 | F. +351 234 919 191 *GPS http://maps.google.com/maps/ms?msa=0msid=202333747613191340808.0004b4b227a6144f0df88 | www.wavecom.pt http://www.wavecom.pt/** http://www.wavecom.pt/* [image: Description: Description: WavecomSignature] http://www.wavecom.pt/pt/wavecom/premios.php [image: Publicity] http://www.wavecom.pt/pt/mail_eventos.php On Thu, Jan 15, 2015 at 4:56 PM, Daniel-Constantin Mierla mico...@gmail.com wrote: Hello, I applied the patch, with some adjustments. Now in master, to be backported to stable branches soon. Cheers, Daniel On 13/01/15 20:16, Nuno Reis wrote: Hi Kristian and Daniel. Kristian, hhanks for you feedback and patch. I'll try your patch here and will let you know the outcome soon. Thanks again guys. Cheers, -- *Nuno Miguel Reis* | *Unified Communication** Systems* M. +351 913907481 | nr...@wavecom.pt WAVECOM-Soluções Rádio, S.A. Cacia Park | Rua do Progresso, Lote 15 3800-639 AVEIRO | Portugal T. +351 309 700 225 | F. +351 234 919 191 *GPS http://maps.google.com/maps/ms?msa=0msid=202333747613191340808.0004b4b227a6144f0df88 | www.wavecom.pt http://www.wavecom.pt/** http://www.wavecom.pt/* [image: Description: Description: WavecomSignature] http://www.wavecom.pt/pt/wavecom/premios.php [image: Publicity] http://www.wavecom.pt/pt/mail_eventos.php On Tue, Jan 13, 2015 at 10:00 AM, Daniel-Constantin Mierla mico...@gmail.com wrote: Hello, thanks for the details and patch. I will try to look at later today. Cheers, Daniel On 13/01/15 08:35, Kristian F. Høgh wrote: Hi, I've been hunting a memory error in publish handling the last couple of days. The error is on our old but good 3.1.x presence server. Using memory debug, I located the memory leak in modules/presence/hash.c, function insert_phtable, line 492 (in trunk): p= (pres_entry_t*)shm_malloc(size); As far I can see there are two errors when deleting publish htable entries 1. When calling delete_phtable pres.event-type is used instead of pres.event-evp-type 2. When creating publish hashtable, p-publ_count is not set. (defaults to 0) In delete_phtable, the following code is present p-publ_count--; if(p-publ_count== 0) p-publ_count is probably decremented to -1 (unless the user have two active dialogs) I attach a patch, which I would carefully test in a test environment :-) Regards, Kristian Høgh Uni-tel On Monday 12 January 2015 15:39:27 Nuno Reis wrote: Hello all. I'm consistently watching a memory increase in kamailio when dealing with PRESENCE events, namely SIP PUBLISH events. The system eventually hangs running out of memory. This behavior is seen at least in kamailio 4.1 and 4.2. I'm currently using the latest stable 4.2.2. If I disable the SIP PUBLISH handling in kamailio i don't observe the issue anymore but as a side effect I don't have presence (name BLFs) also. What do you think can be the right approach here? Should I open an issue in github for this? Should I run kamailio under valgrind for some time? Are there any other possible debug hints here? Please find attached a code snippet with the presence related parts I'm using
Re: [SR-Users] Possible memory leak dealing with presence in kamailio
Nuno Reis writes: Here my publisher is Kamailio itself. Can someone elaborate a bit more on this issue and maybe we can get to bottom of it? when your application issues initial publish request, it does so without SIP-If-Match header. 200 ok from presence server then contains an etag in SIP-ETag header. when your application refreshes the publish, it must place this etag in SIP-If-Match header to prevent presence server from creating a new publication. for subscribes, your application must place in re-subscribe the same event header id param as the previous one had in order for the presence server to know that subscribe was not a new subscription. -- juha ___ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
Re: [SR-Users] Possible memory leak dealing with presence in kamailio
Hello, I applied the patch, with some adjustments. Now in master, to be backported to stable branches soon. Cheers, Daniel On 13/01/15 20:16, Nuno Reis wrote: Hi Kristian and Daniel. Kristian, hhanks for you feedback and patch. I'll try your patch here and will let you know the outcome soon. Thanks again guys. Cheers, -- *Nuno Miguel Reis*| *Unified Communication**Systems* M. +351 913907481 | nr...@wavecom.pt mailto:nr...@wavecom.pt WAVECOM-Soluções Rádio, S.A. Cacia Park | Rua do Progresso, Lote 15 3800-639 AVEIRO | Portugal T. +351 309 700 225 | F. +351 234 919 191 *GPS http://maps.google.com/maps/ms?msa=0msid=202333747613191340808.0004b4b227a6144f0df88 | www.wavecom.pt http://www.wavecom.pt/**http://www.wavecom.pt/* Description: Description: WavecomSignature http://www.wavecom.pt/pt/wavecom/premios.php Publicity http://www.wavecom.pt/pt/mail_eventos.php On Tue, Jan 13, 2015 at 10:00 AM, Daniel-Constantin Mierla mico...@gmail.com mailto:mico...@gmail.com wrote: Hello, thanks for the details and patch. I will try to look at later today. Cheers, Daniel On 13/01/15 08:35, Kristian F. Høgh wrote: Hi, I've been hunting a memory error in publish handling the last couple of days. The error is on our old but good 3.1.x presence server. Using memory debug, I located the memory leak in modules/presence/hash.c, function insert_phtable, line 492 (in trunk): p= (pres_entry_t*)shm_malloc(size); As far I can see there are two errors when deleting publish htable entries 1. When calling delete_phtable pres.event-type is used instead of pres.event-evp-type 2. When creating publish hashtable, p-publ_count is not set. (defaults to 0) In delete_phtable, the following code is present p-publ_count--; if(p-publ_count== 0) p-publ_count is probably decremented to -1 (unless the user have two active dialogs) I attach a patch, which I would carefully test in a test environment :-) Regards, Kristian Høgh Uni-tel On Monday 12 January 2015 15:39:27 Nuno Reis wrote: Hello all. I'm consistently watching a memory increase in kamailio when dealing with PRESENCE events, namely SIP PUBLISH events. The system eventually hangs running out of memory. This behavior is seen at least in kamailio 4.1 and 4.2. I'm currently using the latest stable 4.2.2. If I disable the SIP PUBLISH handling in kamailio i don't observe the issue anymore but as a side effect I don't have presence (name BLFs) also. What do you think can be the right approach here? Should I open an issue in github for this? Should I run kamailio under valgrind for some time? Are there any other possible debug hints here? Please find attached a code snippet with the presence related parts I'm using right now. Looking forward to hear from you. Best Regards, -- *Nuno Miguel Reis* | *Unified Communication** Systems* M. +351 913907481 tel:%2B351%20913907481 | nr...@wavecom.pt mailto:nr...@wavecom.pt WAVECOM-Soluções Rádio, S.A. Cacia Park | Rua do Progresso, Lote 15 3800-639 AVEIRO | Portugal T. +351 309 700 225 tel:%2B351%20309%20700%20225 | F. +351 234 919 191 *GPS http://maps.google.com/maps/ms?msa=0msid=202333747613191340808.0004b4b227a6144f0df88 http://maps.google.com/maps/ms?msa=0msid=202333747613191340808.0004b4b227a6144f0df88 | www.wavecom.pt http://www.wavecom.pt http://www.wavecom.pt/ http://www.wavecom.pt/** http://www.wavecom.pt/ http://www.wavecom.pt/* [image: Description: Description: WavecomSignature] http://www.wavecom.pt/pt/wavecom/premios.php http://www.wavecom.pt/pt/wavecom/premios.php [image: Publicity] http://www.wavecom.pt/pt/mail_eventos.php http://www.wavecom.pt/pt/mail_eventos.php ___ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org mailto:sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users -- Daniel-Constantin Mierla http://twitter.com/#!/miconda http://twitter.com/#%21/miconda - http://www.linkedin.com/in/miconda ___ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org mailto:sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users -- Daniel-Constantin Mierla http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda
Re: [SR-Users] Possible memory leak dealing with presence in kamailio
Hi Kristian and Daniel. Kristian, hhanks for you feedback and patch. I'll try your patch here and will let you know the outcome soon. Thanks again guys. Cheers, -- *Nuno Miguel Reis* | *Unified Communication** Systems* M. +351 913907481 | nr...@wavecom.pt WAVECOM-Soluções Rádio, S.A. Cacia Park | Rua do Progresso, Lote 15 3800-639 AVEIRO | Portugal T. +351 309 700 225 | F. +351 234 919 191 *GPS http://maps.google.com/maps/ms?msa=0msid=202333747613191340808.0004b4b227a6144f0df88 | www.wavecom.pt http://www.wavecom.pt/** http://www.wavecom.pt/* [image: Description: Description: WavecomSignature] http://www.wavecom.pt/pt/wavecom/premios.php [image: Publicity] http://www.wavecom.pt/pt/mail_eventos.php On Tue, Jan 13, 2015 at 10:00 AM, Daniel-Constantin Mierla mico...@gmail.com wrote: Hello, thanks for the details and patch. I will try to look at later today. Cheers, Daniel On 13/01/15 08:35, Kristian F. Høgh wrote: Hi, I've been hunting a memory error in publish handling the last couple of days. The error is on our old but good 3.1.x presence server. Using memory debug, I located the memory leak in modules/presence/hash.c, function insert_phtable, line 492 (in trunk): p= (pres_entry_t*)shm_malloc(size); As far I can see there are two errors when deleting publish htable entries 1. When calling delete_phtable pres.event-type is used instead of pres.event-evp-type 2. When creating publish hashtable, p-publ_count is not set. (defaults to 0) In delete_phtable, the following code is present p-publ_count--; if(p-publ_count== 0) p-publ_count is probably decremented to -1 (unless the user have two active dialogs) I attach a patch, which I would carefully test in a test environment :-) Regards, Kristian Høgh Uni-tel On Monday 12 January 2015 15:39:27 Nuno Reis wrote: Hello all. I'm consistently watching a memory increase in kamailio when dealing with PRESENCE events, namely SIP PUBLISH events. The system eventually hangs running out of memory. This behavior is seen at least in kamailio 4.1 and 4.2. I'm currently using the latest stable 4.2.2. If I disable the SIP PUBLISH handling in kamailio i don't observe the issue anymore but as a side effect I don't have presence (name BLFs) also. What do you think can be the right approach here? Should I open an issue in github for this? Should I run kamailio under valgrind for some time? Are there any other possible debug hints here? Please find attached a code snippet with the presence related parts I'm using right now. Looking forward to hear from you. Best Regards, -- *Nuno Miguel Reis* | *Unified Communication** Systems* M. +351 913907481 | nr...@wavecom.pt WAVECOM-Soluções Rádio, S.A. Cacia Park | Rua do Progresso, Lote 15 3800-639 AVEIRO | Portugal T. +351 309 700 225 | F. +351 234 919 191 *GPS http://maps.google.com/maps/ms?msa=0msid=202333747613191340808.0004b4b227a6144f0df88 http://maps.google.com/maps/ms?msa=0msid=202333747613191340808.0004b4b227a6144f0df88 | www.wavecom.pt http://www.wavecom.pt/ http://www.wavecom.pt/** http://www.wavecom.pt/ http://www.wavecom.pt/* [image: Description: Description: WavecomSignature] http://www.wavecom.pt/pt/wavecom/premios.php http://www.wavecom.pt/pt/wavecom/premios.php [image: Publicity] http://www.wavecom.pt/pt/mail_eventos.php http://www.wavecom.pt/pt/mail_eventos.php ___ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing listsr-us...@lists.sip-router.orghttp://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users -- Daniel-Constantin Mierlahttp://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda ___ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users ___ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
Re: [SR-Users] Possible memory leak dealing with presence in kamailio
Hello, thanks for the details and patch. I will try to look at later today. Cheers, Daniel On 13/01/15 08:35, Kristian F. Høgh wrote: Hi, I've been hunting a memory error in publish handling the last couple of days. The error is on our old but good 3.1.x presence server. Using memory debug, I located the memory leak in modules/presence/hash.c, function insert_phtable, line 492 (in trunk): p= (pres_entry_t*)shm_malloc(size); As far I can see there are two errors when deleting publish htable entries 1. When calling delete_phtable pres.event-type is used instead of pres.event-evp-type 2. When creating publish hashtable, p-publ_count is not set. (defaults to 0) In delete_phtable, the following code is present p-publ_count--; if(p-publ_count== 0) p-publ_count is probably decremented to -1 (unless the user have two active dialogs) I attach a patch, which I would carefully test in a test environment :-) Regards, Kristian Høgh Uni-tel On Monday 12 January 2015 15:39:27 Nuno Reis wrote: Hello all. I'm consistently watching a memory increase in kamailio when dealing with PRESENCE events, namely SIP PUBLISH events. The system eventually hangs running out of memory. This behavior is seen at least in kamailio 4.1 and 4.2. I'm currently using the latest stable 4.2.2. If I disable the SIP PUBLISH handling in kamailio i don't observe the issue anymore but as a side effect I don't have presence (name BLFs) also. What do you think can be the right approach here? Should I open an issue in github for this? Should I run kamailio under valgrind for some time? Are there any other possible debug hints here? Please find attached a code snippet with the presence related parts I'm using right now. Looking forward to hear from you. Best Regards, -- *Nuno Miguel Reis* | *Unified Communication** Systems* M. +351 913907481 | nr...@wavecom.pt WAVECOM-Soluções Rádio, S.A. Cacia Park | Rua do Progresso, Lote 15 3800-639 AVEIRO | Portugal T. +351 309 700 225 | F. +351 234 919 191 *GPS http://maps.google.com/maps/ms?msa=0msid=202333747613191340808.0004b4b227a6144f0df88 | www.wavecom.pt http://www.wavecom.pt/** http://www.wavecom.pt/* [image: Description: Description: WavecomSignature] http://www.wavecom.pt/pt/wavecom/premios.php [image: Publicity] http://www.wavecom.pt/pt/mail_eventos.php ___ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users -- Daniel-Constantin Mierla http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda ___ SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
Re: [SR-Users] Possible memory leak dealing with presence in kamailio
Hi,
I've been hunting a memory error in publish handling the last couple of days.
The error is on our old but good 3.1.x presence server.
Using memory debug, I located the memory leak in modules/presence/hash.c,
function insert_phtable, line 492 (in trunk):
p= (pres_entry_t*)shm_malloc(size);
As far I can see there are two errors when deleting publish htable entries
1. When calling delete_phtable pres.event-type is used instead of
pres.event-evp-type
2. When creating publish hashtable, p-publ_count is not set. (defaults to 0)
In delete_phtable, the following code is present
p-publ_count--;
if(p-publ_count== 0)
p-publ_count is probably decremented to -1 (unless the user have two active
dialogs)
I attach a patch, which I would carefully test in a test environment :-)
Regards,
Kristian Høgh
Uni-tel
On Monday 12 January 2015 15:39:27 Nuno Reis wrote:
Hello all.
I'm consistently watching a memory increase in kamailio when dealing with
PRESENCE events, namely SIP PUBLISH events. The system eventually hangs
running out of memory.
This behavior is seen at least in kamailio 4.1 and 4.2. I'm currently using
the latest stable 4.2.2.
If I disable the SIP PUBLISH handling in kamailio i don't observe the issue
anymore but as a side effect I don't have presence (name BLFs) also.
What do you think can be the right approach here? Should I open an issue in
github for this? Should I run kamailio under valgrind for some time? Are
there any other possible debug hints here?
Please find attached a code snippet with the presence related parts I'm
using right now.
Looking forward to hear from you.
Best Regards,
--
*Nuno Miguel Reis* | *Unified Communication** Systems*
M. +351 913907481 | nr...@wavecom.pt
WAVECOM-Soluções Rádio, S.A.
Cacia Park | Rua do Progresso, Lote 15
3800-639 AVEIRO | Portugal
T. +351 309 700 225 | F. +351 234 919 191
*GPS
http://maps.google.com/maps/ms?msa=0msid=202333747613191340808.0004b4b227a6144f0df88
| www.wavecom.pt http://www.wavecom.pt/** http://www.wavecom.pt/*
[image: Description: Description: WavecomSignature]
http://www.wavecom.pt/pt/wavecom/premios.php
[image: Publicity] http://www.wavecom.pt/pt/mail_eventos.php
diff --git a/modules/presence/hash.c b/modules/presence/hash.c
index 6e85d37..3bc1f1c 100644
--- a/modules/presence/hash.c
+++ b/modules/presence/hash.c
@@ -544,6 +544,7 @@ int insert_phtable(str* pres_uri, int event, char* sphere)
p-next= pres_htable[hash_code].entries-next;
pres_htable[hash_code].entries-next= p;
+ p-publ_count=1;
lock_release(pres_htable[hash_code].lock);
return 0;
diff --git a/modules/presence/publish.c b/modules/presence/publish.c
index 46ec334..dfe8f31 100644
--- a/modules/presence/publish.c
+++ b/modules/presence/publish.c
@@ -147,7 +147,7 @@ void msg_presentity_clean(unsigned int ticks,void *param)
}
/* delete from hash table */
- if(publ_cache_enabled delete_phtable(uri, pres.event-type) 0)
+ if(publ_cache_enabled delete_phtable(uri, pres.event-evp-type) 0)
{
LM_ERR(deleting from pres hash table\n);
goto error;
___
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
sr-users@lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
