[web2py] Re: Web2py scheduler linked postgres processes consuming too much memory
you're seeing a totally normal thing: postgresql spawns a different process for each connection. Those processes in linux are fork()s, so even if you SEEM to notice a skyrocket in memory utilization summing all different processes (e.g. in "top"), in reality it's not sucking up every bit of RAM of your server. I'm sure this is not your case, but if you see postgresql processes comsuming memory, the leak in postgresql processes CAN'T be generated by web2py in any circumnstance. Unless you tinkered a lot with postgresql.conf (which is pretty conservative in all distros) you won't ever see postgresql leaking. On Sunday, July 17, 2016 at 5:50:04 AM UTC+2, Abhishek Ram wrote: > > Hi, > > I am using the web2py scheduler to run some background jobs. I am starting > the scheduler using systemd and once started it also spawns two postgres > processes as that is my DB. Now the problem is that the postgres processes > continue to consume more and more memory even when no process is running. > > So my question is what is causing the scheduler process to do this? > > P.S. The postgres processes linked to the main web2py process do not > exhibit this behaviour. > > Regards > Abhishek Ram > -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[web2py] Re: Web2py scheduler linked postgres processes consuming too much memory
Well the memory leak was from the postgres processes spawned by the web2py scheduler process. I am sure about this as I have seen TOP and confirmed that it was these processes that were consuming the memory. I even tried with a basic app and the postgres processes continued to consume memory. So finally I shifted to mysql and memory consumption is now stable. On Tuesday, July 19, 2016 at 12:24:27 AM UTC+5:30, Niphlod wrote: > > you're seeing a totally normal thing: postgresql spawns a different > process for each connection. > Those processes in linux are fork()s, so even if you SEEM to notice a > skyrocket in memory utilization summing all different processes (e.g. in > "top"), in reality it's not sucking up every bit of RAM of your server. > > I'm sure this is not your case, but if you see postgresql processes > comsuming memory, the leak in postgresql processes CAN'T be generated by > web2py in any circumnstance. Unless you tinkered a lot with postgresql.conf > (which is pretty conservative in all distros) you won't ever see postgresql > leaking. > > On Sunday, July 17, 2016 at 5:50:04 AM UTC+2, Abhishek Ram wrote: >> >> Hi, >> >> I am using the web2py scheduler to run some background jobs. I am >> starting the scheduler using systemd and once started it also spawns two >> postgres processes as that is my DB. Now the problem is that the postgres >> processes continue to consume more and more memory even when no process is >> running. >> >> So my question is what is causing the scheduler process to do this? >> >> P.S. The postgres processes linked to the main web2py process do not >> exhibit this behaviour. >> >> Regards >> Abhishek Ram >> > -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[web2py] Re: Web2py scheduler linked postgres processes consuming too much memory
what driver are you using ? it's rather strange that postrgresql acts up and eat all the available memory, if you didn't tinker with the defaults settings of postgresql... On Wednesday, July 27, 2016 at 5:09:34 AM UTC+2, Abhishek Ram wrote: > > Well the memory leak was from the postgres processes spawned by the web2py > scheduler process. I am sure about this as I have seen TOP and confirmed > that it was these processes that were consuming the memory. I even tried > with a basic app and the postgres processes continued to consume memory. So > finally I shifted to mysql and memory consumption is now stable. > > On Tuesday, July 19, 2016 at 12:24:27 AM UTC+5:30, Niphlod wrote: >> >> you're seeing a totally normal thing: postgresql spawns a different >> process for each connection. >> Those processes in linux are fork()s, so even if you SEEM to notice a >> skyrocket in memory utilization summing all different processes (e.g. in >> "top"), in reality it's not sucking up every bit of RAM of your server. >> >> I'm sure this is not your case, but if you see postgresql processes >> comsuming memory, the leak in postgresql processes CAN'T be generated by >> web2py in any circumnstance. Unless you tinkered a lot with postgresql.conf >> (which is pretty conservative in all distros) you won't ever see postgresql >> leaking. >> >> On Sunday, July 17, 2016 at 5:50:04 AM UTC+2, Abhishek Ram wrote: >>> >>> Hi, >>> >>> I am using the web2py scheduler to run some background jobs. I am >>> starting the scheduler using systemd and once started it also spawns two >>> postgres processes as that is my DB. Now the problem is that the postgres >>> processes continue to consume more and more memory even when no process is >>> running. >>> >>> So my question is what is causing the scheduler process to do this? >>> >>> P.S. The postgres processes linked to the main web2py process do not >>> exhibit this behaviour. >>> >>> Regards >>> Abhishek Ram >>> >> -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[web2py] Re: Web2py scheduler linked postgres processes consuming too much memory
I am using the default driver pg8000 On Thursday, July 28, 2016 at 7:47:41 PM UTC+5:30, Niphlod wrote: > > what driver are you using ? it's rather strange that postrgresql acts up > and eat all the available memory, if you didn't tinker with the defaults > settings of postgresql... > > On Wednesday, July 27, 2016 at 5:09:34 AM UTC+2, Abhishek Ram wrote: >> >> Well the memory leak was from the postgres processes spawned by the >> web2py scheduler process. I am sure about this as I have seen TOP and >> confirmed that it was these processes that were consuming the memory. I >> even tried with a basic app and the postgres processes continued to consume >> memory. So finally I shifted to mysql and memory consumption is now stable. >> >> On Tuesday, July 19, 2016 at 12:24:27 AM UTC+5:30, Niphlod wrote: >>> >>> you're seeing a totally normal thing: postgresql spawns a different >>> process for each connection. >>> Those processes in linux are fork()s, so even if you SEEM to notice a >>> skyrocket in memory utilization summing all different processes (e.g. in >>> "top"), in reality it's not sucking up every bit of RAM of your server. >>> >>> I'm sure this is not your case, but if you see postgresql processes >>> comsuming memory, the leak in postgresql processes CAN'T be generated by >>> web2py in any circumnstance. Unless you tinkered a lot with postgresql.conf >>> (which is pretty conservative in all distros) you won't ever see postgresql >>> leaking. >>> >>> On Sunday, July 17, 2016 at 5:50:04 AM UTC+2, Abhishek Ram wrote: Hi, I am using the web2py scheduler to run some background jobs. I am starting the scheduler using systemd and once started it also spawns two postgres processes as that is my DB. Now the problem is that the postgres processes continue to consume more and more memory even when no process is running. So my question is what is causing the scheduler process to do this? P.S. The postgres processes linked to the main web2py process do not exhibit this behaviour. Regards Abhishek Ram >>> -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[web2py] Re: Web2py scheduler linked postgres processes consuming too much memory
could you please try with psycopg2 ? On Thursday, July 28, 2016 at 6:53:10 PM UTC+2, Abhishek Ram wrote: > > I am using the default driver pg8000 > > On Thursday, July 28, 2016 at 7:47:41 PM UTC+5:30, Niphlod wrote: >> >> what driver are you using ? it's rather strange that postrgresql acts up >> and eat all the available memory, if you didn't tinker with the defaults >> settings of postgresql... >> >> On Wednesday, July 27, 2016 at 5:09:34 AM UTC+2, Abhishek Ram wrote: >>> >>> Well the memory leak was from the postgres processes spawned by the >>> web2py scheduler process. I am sure about this as I have seen TOP and >>> confirmed that it was these processes that were consuming the memory. I >>> even tried with a basic app and the postgres processes continued to consume >>> memory. So finally I shifted to mysql and memory consumption is now stable. >>> >>> On Tuesday, July 19, 2016 at 12:24:27 AM UTC+5:30, Niphlod wrote: you're seeing a totally normal thing: postgresql spawns a different process for each connection. Those processes in linux are fork()s, so even if you SEEM to notice a skyrocket in memory utilization summing all different processes (e.g. in "top"), in reality it's not sucking up every bit of RAM of your server. I'm sure this is not your case, but if you see postgresql processes comsuming memory, the leak in postgresql processes CAN'T be generated by web2py in any circumnstance. Unless you tinkered a lot with postgresql.conf (which is pretty conservative in all distros) you won't ever see postgresql leaking. On Sunday, July 17, 2016 at 5:50:04 AM UTC+2, Abhishek Ram wrote: > > Hi, > > I am using the web2py scheduler to run some background jobs. I am > starting the scheduler using systemd and once started it also spawns two > postgres processes as that is my DB. Now the problem is that the postgres > processes continue to consume more and more memory even when no process > is > running. > > So my question is what is causing the scheduler process to do this? > > P.S. The postgres processes linked to the main web2py process do not > exhibit this behaviour. > > Regards > Abhishek Ram > -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[web2py] Re: Web2py scheduler linked postgres processes consuming too much memory
The pg8000 driver has problems. Sorry. We should remove it from web2py. On Thursday, 28 July 2016 11:53:10 UTC-5, Abhishek Ram wrote: > > I am using the default driver pg8000 > > On Thursday, July 28, 2016 at 7:47:41 PM UTC+5:30, Niphlod wrote: >> >> what driver are you using ? it's rather strange that postrgresql acts up >> and eat all the available memory, if you didn't tinker with the defaults >> settings of postgresql... >> >> On Wednesday, July 27, 2016 at 5:09:34 AM UTC+2, Abhishek Ram wrote: >>> >>> Well the memory leak was from the postgres processes spawned by the >>> web2py scheduler process. I am sure about this as I have seen TOP and >>> confirmed that it was these processes that were consuming the memory. I >>> even tried with a basic app and the postgres processes continued to consume >>> memory. So finally I shifted to mysql and memory consumption is now stable. >>> >>> On Tuesday, July 19, 2016 at 12:24:27 AM UTC+5:30, Niphlod wrote: you're seeing a totally normal thing: postgresql spawns a different process for each connection. Those processes in linux are fork()s, so even if you SEEM to notice a skyrocket in memory utilization summing all different processes (e.g. in "top"), in reality it's not sucking up every bit of RAM of your server. I'm sure this is not your case, but if you see postgresql processes comsuming memory, the leak in postgresql processes CAN'T be generated by web2py in any circumnstance. Unless you tinkered a lot with postgresql.conf (which is pretty conservative in all distros) you won't ever see postgresql leaking. On Sunday, July 17, 2016 at 5:50:04 AM UTC+2, Abhishek Ram wrote: > > Hi, > > I am using the web2py scheduler to run some background jobs. I am > starting the scheduler using systemd and once started it also spawns two > postgres processes as that is my DB. Now the problem is that the postgres > processes continue to consume more and more memory even when no process > is > running. > > So my question is what is causing the scheduler process to do this? > > P.S. The postgres processes linked to the main web2py process do not > exhibit this behaviour. > > Regards > Abhishek Ram > -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[web2py] Re: Web2py scheduler linked postgres processes consuming too much memory
I have tested it and I can now confirm that the issue is with pg8000. I think its a good idea to mention this in the WIKI. On Friday, July 29, 2016 at 12:12:07 AM UTC+5:30, Niphlod wrote: > > could you please try with psycopg2 ? > > On Thursday, July 28, 2016 at 6:53:10 PM UTC+2, Abhishek Ram wrote: >> >> I am using the default driver pg8000 >> >> On Thursday, July 28, 2016 at 7:47:41 PM UTC+5:30, Niphlod wrote: >>> >>> what driver are you using ? it's rather strange that postrgresql acts up >>> and eat all the available memory, if you didn't tinker with the defaults >>> settings of postgresql... >>> >>> On Wednesday, July 27, 2016 at 5:09:34 AM UTC+2, Abhishek Ram wrote: Well the memory leak was from the postgres processes spawned by the web2py scheduler process. I am sure about this as I have seen TOP and confirmed that it was these processes that were consuming the memory. I even tried with a basic app and the postgres processes continued to consume memory. So finally I shifted to mysql and memory consumption is now stable. On Tuesday, July 19, 2016 at 12:24:27 AM UTC+5:30, Niphlod wrote: > > you're seeing a totally normal thing: postgresql spawns a different > process for each connection. > Those processes in linux are fork()s, so even if you SEEM to notice a > skyrocket in memory utilization summing all different processes (e.g. in > "top"), in reality it's not sucking up every bit of RAM of your server. > > I'm sure this is not your case, but if you see postgresql processes > comsuming memory, the leak in postgresql processes CAN'T be generated by > web2py in any circumnstance. Unless you tinkered a lot with > postgresql.conf > (which is pretty conservative in all distros) you won't ever see > postgresql > leaking. > > On Sunday, July 17, 2016 at 5:50:04 AM UTC+2, Abhishek Ram wrote: >> >> Hi, >> >> I am using the web2py scheduler to run some background jobs. I am >> starting the scheduler using systemd and once started it also spawns two >> postgres processes as that is my DB. Now the problem is that the >> postgres >> processes continue to consume more and more memory even when no process >> is >> running. >> >> So my question is what is causing the scheduler process to do this? >> >> P.S. The postgres processes linked to the main web2py process do not >> exhibit this behaviour. >> >> Regards >> Abhishek Ram >> > -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
