Thanks. So the key point for me is to just use engine.dispose() (I don't want to dig into gunicorn preload, this seems much cleaner).
About explicit manager: in practice it's as simple as 1. enabling it in models/__init__.py 2. using "with request.tm" everywhere instead of "with transaction.manager", right? > This is more likely to be an issue in web requests where your entire request > lifecycle is not actually protected by a transaction. About your final sentence, I'm not sure I understand it. If I'm always using request.dbsession, by definition I'm protected, to a request's lifecycle am I not? Zsolt On 2 April 2018 at 21:47, Michael Merickel <mmeri...@gmail.com> wrote: > The forking issue is likely because you're using a connection pool and so > once a connection is opened at config-time, even though the session is > properly closed the connection is just returned to the pool. The pool here > is shared across the fork which is bad. The basic solution here is to add > some sort of pre-fork hook that closes out every existing connection in the > pool. I personally have not used forking wsgi servers in production and thus > have not needed to deal with this problem but I'm pretty sure my advice hits > at the core issue you're experiencing. Calling dispose() is one way to make > sure that the underlying connection is closed which is why it's working for > you. If I recall, gunicorn has some way to fork before config is called such > that each subprocess is loaded independently which can help with this as > well... If memory serves it was something like turning off the preloading > feature. > > The explicit manager is about weeding out situations where you use a > connection after it's closed [1]. Basically without it you could use > something joined to the "tm" after the "with" block is done and you might > not see an error. This is more likely to be an issue in web requests where > your entire request lifecycle is not actually protected by a transaction. > > [1] > https://docs.pylonsproject.org/projects/pyramid-tm/en/latest/#custom-transaction-managers > > On Mon, Apr 2, 2018 at 2:21 PM, Zsolt Ero <zsolt....@gmail.com> wrote: >> >> Michael, I've updated the code to your recommendation. >> >> https://github.com/hyperknot/pyramid_connections_bug >> >> It still requires the explicit engine.dispose() line, otherwise it >> does bring the connection to the forked processes. >> >> This is with and without the explicit manager. What does the explicit >> manager protect us from? >> >> Zsolt >> >> >> On 2 April 2018 at 20:11, Michael Merickel <mmeri...@gmail.com> wrote: >> > Using the pyramid-cookiecutter-alchemy setup you can access config data >> > at >> > config time using a pattern like this: >> > >> > from transaction import TransactionManager >> > >> > from myapp.models import get_tm_session >> > >> > def main(global_config, **settings): >> > config = Configurator(settings=settings) >> > config.include('myapp.models') >> > >> > tm = TransactionManager(explicit=True) >> > with tm: >> > dbsession = get_tm_session(config.registry['dbsession_factory'], >> > tm) >> > ... # do queries and stuff >> > >> > return config.make_wsgi_app() >> > >> > This will properly handle the lifecycle of a session for you, same as >> > when >> > serving a request. Be sure if you keep any objects around that you >> > expunge >> > them from the session and re-attach/merge them into any session where >> > you >> > use them.. ORM objects are only valid on the dbsession they were loaded >> > from. >> > >> > - Michael >> > >> > >> > On Mon, Apr 2, 2018 at 11:59 AM, Zsolt Ero <zsolt....@gmail.com> wrote: >> >> >> >> OK, I agree with that. Still, storing config values in the database is >> >> a common pattern for medium to large web apps, so it at least makes >> >> sense to have some kind of resource about how to do it. I hope that if >> >> nothing else at least this thread will be useful for someone in the >> >> future. >> >> >> >> -- >> >> You received this message because you are subscribed to the Google >> >> Groups >> >> "pylons-discuss" group. >> >> To unsubscribe from this group and stop receiving emails from it, send >> >> an >> >> email to pylons-discuss+unsubscr...@googlegroups.com. >> >> To post to this group, send email to pylons-discuss@googlegroups.com. >> >> To view this discussion on the web visit >> >> >> >> https://groups.google.com/d/msgid/pylons-discuss/CAKw-smBuH5hKGpNgs-6dYkNiAU%3DGGUUOF7aLHAzHe0AE25T7qg%40mail.gmail.com. >> >> For more options, visit https://groups.google.com/d/optout. >> > >> > >> > -- >> > You received this message because you are subscribed to a topic in the >> > Google Groups "pylons-discuss" group. >> > To unsubscribe from this topic, visit >> > >> > https://groups.google.com/d/topic/pylons-discuss/_MJflNUcjdg/unsubscribe. >> > To unsubscribe from this group and all its topics, send an email to >> > pylons-discuss+unsubscr...@googlegroups.com. >> > To post to this group, send email to pylons-discuss@googlegroups.com. >> > To view this discussion on the web visit >> > >> > https://groups.google.com/d/msgid/pylons-discuss/CAKdhhwGoPbgM7gDWBLxdanT%3DhcJQR8sT01E2xwWFxw1zxpCD1A%40mail.gmail.com. >> > >> > For more options, visit https://groups.google.com/d/optout. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "pylons-discuss" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to pylons-discuss+unsubscr...@googlegroups.com. >> To post to this group, send email to pylons-discuss@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/pylons-discuss/CAKw-smBMmRqu2ZTPSp%3Djh%2BY0RriEwvxgdYQnwq--k4abeUkd-g%40mail.gmail.com. >> For more options, visit https://groups.google.com/d/optout. > > > -- > You received this message because you are subscribed to a topic in the > Google Groups "pylons-discuss" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/pylons-discuss/_MJflNUcjdg/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > pylons-discuss+unsubscr...@googlegroups.com. > To post to this group, send email to pylons-discuss@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/pylons-discuss/CAKdhhwGFjP7-aYzz4NEewWv1kdbMxhH25hrX4JC3oR4ZizqYeg%40mail.gmail.com. > > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "pylons-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discuss+unsubscr...@googlegroups.com. To post to this group, send email to pylons-discuss@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/CAKw-smDuP2324T0GLi-J83UH-a5mVVvtPC3HyUAgB65aRSGQ5Q%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
