On Monday, April 2, 2018 at 12:09:26 PM UTC-4, Zsolt Ero wrote:
> So if I understand right, the engine in a gunicorn / process worker > Pyramid app is global / shared between all processes, right? And at > the time of forking, there should be no dbsessions being active and no > connections in the connection pool, right? > Sort of. I'll try to explain this simply and using terms closer to you. The engine is global but you DON'T want to share it between processes because of Python's copy-on-write behavior and how connections are handled as file-descriptors. When each process creates a new connection (or otherwise updates the pool), the pool is basically copied into a per-worker variable then updated. If you have active connections in the pool *before* the fork, everyone gets a copy of that connection and thinks it is theirs -- but if the pool is empty, everyone gets an empty pool. > Where is the forking happening in a normal Pyramid app? After the main > function finishes / "return config.make_wsgi_app()"? > Somewhere after there, i believe. It depends on the server. uwsgi and unicorn do it in different places. i think waitress essentially does it too. gunicorn offers a post_fork hook (http://docs.gunicorn.org/en/stable/settings.html) which you can use to trigger a dispose. If you connect within the __main__ to pull config settings out of the db, you generally want to immediately drop the connection. Doing it within your startup routine and before the fork is fine - but you can also do it after the fork. All that matters is that the engine is disposed before you use it. I'm checking with SqlAlchemy, but it looks like `dispose` doesn't explicitly close the session - it just removes it from the connection pool which will effectively close it during garbage collection. calling close()+dispose() in your startup should be fine. there's another technique to protect against forks that mike (bayer of sqlalchemy) has been using in openstack and may bring to the main project, but I'm not sure if/when that will happen. if you think you may jump between platforms and use multiple libraries/patterns that aren't fork-safe, I wrote a library that turns uwsgi & gunicorn events into subscribers (https://github.com/jvanasco/pyramid_forksafe) that make the cleanup work a little more portable. We use sqlalchemy, pymongo and pycrypto which are all not forksafe. -- You received this message because you are subscribed to the Google Groups "pylons-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discuss+unsubscr...@googlegroups.com. To post to this group, send email to pylons-discuss@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/82a9fae4-2fe5-452e-8388-68007f8c2c6a%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
