Damian wrote: > Hi, > > Every few days, when we experience higher loads we get sqlalchemy's > > TimeoutError: QueuePool limit of size 5 overflow 10 reached, > connection timed out, timeout 30 > > Along with that I see an increase in (2-3 a minute): > > (104)Connection reset by peer: core_output_filter: writing data to the > network > > and > > (32)Broken pipe: core_output_filter: writing data to the network > > in my apache error logs. > > Having checked over my pylons code a few times, the Session.remove() > should always be called. I'm worried that the broken pipe or > connection reset by peer mean that remove isn't being called.
sounds more like the higher load means more concurrent DB activity, PG then takes more time to return results, more reqs then come in, connections aren't available since they're all busy. AFAIK the broken pipe stuff doesn't kill off the Python threads in the middle of their work...if they threw an exception, hopefully you have Session.remove() happening within a finally:. Since you're on PG, just to a "ps" listing on your database machine or better yet use "pgtop" and you'll see just what happens when a spike comes in. > > The server is running mod_wsgi with apaches mpm_worker with the > following config: > > <IfModule mpm_worker_module> > StartServers 16 > MaxClients 480 > MinSpareThreads 50 > MaxSpareThreads 300 > ThreadsPerChild 30 > MaxRequestsPerChild 0 > </IfModule> > > and using mod_wsgi's daemon mode: > > WSGIDaemonProcess somename user=www-data group=www-data processes=4 > threads=32 > > Is this somehow overkill? The server is a well speced quad core with > 8 gigs of ram and fast hard drives. > > It also runs the database server (postgres). > > Has anyone else experienced this kind of problem? I've cross posted > this to both the mod_wsgi and sqlalchemy mailing lists - hope that's > ok as I believe this may be relevant to both groups. > > Thanks, > Damian > -- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To post to this group, send email to sqlalch...@googlegroups.com. > To unsubscribe from this group, send email to > sqlalchemy+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/sqlalchemy?hl=en. > > >
-- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
