Re: [pylons-discuss] Pyramid + SQLAlchemy + PostgreSQL: idle connections

The best answer is to use "with request.tm" in your console script as it will then use the same manager config as your requests... which I suggest to be explicit of course. We need to update the cookiecutter with a better example here. The initialize_db script is a bad example because it is in

Re: [pylons-discuss] Pyramid + SQLAlchemy + PostgreSQL: idle connections

Thanks for the detailed explanation. Yes I was talking about console scripts. 1. First, I had my console scripts based on initializedb.py (with no explicit manager in models/__init__.py)

Re: [pylons-discuss] Pyramid + SQLAlchemy + PostgreSQL: idle connections

You almost never want to use "with request.tm". This cannot be nested with another call to request.tm.begin() and pyramid_tm already does request.tm.begin() for you. You can basically think of pyramid_tm as doing a "with request.tm" in a tween around all of your views except for some exception

Re: [pylons-discuss] Pyramid + SQLAlchemy + PostgreSQL: idle connections

I confirmed with mike @ sqlalchemy. calling `dispose()` will not explicitly `close()` connections that are already checked out. It just gets rid of the the connection pool instance, so you have fresh connections in the child processes-- the pre-fork connections themselves will be garbage

Re: [pylons-discuss] Pyramid + SQLAlchemy + PostgreSQL: idle connections

Thanks. So the key point for me is to just use engine.dispose() (I don't want to dig into gunicorn preload, this seems much cleaner). About explicit manager: in practice it's as simple as 1. enabling it in models/__init__.py 2. using "with request.tm" everywhere instead of "with

Re: [pylons-discuss] Pyramid + SQLAlchemy + PostgreSQL: idle connections

The forking issue is likely because you're using a connection pool and so once a connection is opened at config-time, even though the session is properly closed the connection is just returned to the pool. The pool here is shared across the fork which is bad. The basic solution here is to add some

Re: [pylons-discuss] Pyramid + SQLAlchemy + PostgreSQL: idle connections

Michael, I've updated the code to your recommendation. https://github.com/hyperknot/pyramid_connections_bug It still requires the explicit engine.dispose() line, otherwise it does bring the connection to the forked processes. This is with and without the explicit manager. What does the explicit

Re: [pylons-discuss] Pyramid + SQLAlchemy + PostgreSQL: idle connections

Using the pyramid-cookiecutter-alchemy setup you can access config data at config time using a pattern like this: from transaction import TransactionManager from myapp.models import get_tm_session def main(global_config, **settings): config = Configurator(settings=settings)

Re: [pylons-discuss] Pyramid + SQLAlchemy + PostgreSQL: idle connections

On Monday, April 2, 2018 at 12:09:26 PM UTC-4, Zsolt Ero wrote: > So if I understand right, the engine in a gunicorn / process worker > Pyramid app is global / shared between all processes, right? And at > the time of forking, there should be no dbsessions being active and no > connections

Re: [pylons-discuss] Pyramid + SQLAlchemy + PostgreSQL: idle connections

OK, I agree with that. Still, storing config values in the database is a common pattern for medium to large web apps, so it at least makes sense to have some kind of resource about how to do it. I hope that if nothing else at least this thread will be useful for someone in the future. -- You

Re: [pylons-discuss] Pyramid + SQLAlchemy + PostgreSQL: idle connections

The fool proof way would be to not use dbsession inside of your config cycle. You are more than welcome to implement whatever functionality you need though, but it's up to you to do so safely. I don't believe it is worth the time for us to implement/test/and maintain that code as part of

Re: [pylons-discuss] Pyramid + SQLAlchemy + PostgreSQL: idle connections

Wouldn't the second / "foolproof" way described here be a good choice for the default Pyramid implementation? Just as a safety measure. http://docs.sqlalchemy.org/en/latest/core/pooling.html#using-connection-pools-with-multiprocessing On 2 April 2018 at 18:32, Jonathan Vanasco

Re: [pylons-discuss] Pyramid + SQLAlchemy + PostgreSQL: idle connections

On Monday, April 2, 2018 at 12:07:14 PM UTC-4, Bert JW Regeer wrote: > > > This is only required if you are not using pyramid_tm. If you are using > pyramid_tm which is what the sqlalchemy cookie cutter does, you do NOT need > to add this. > Thanks, Bert. I stand corrected. pyarmid_tm has a

Re: [pylons-discuss] Pyramid + SQLAlchemy + PostgreSQL: idle connections

I've been looking into it a bit more. I _think_ when a dbsession is deleted / garbage collected, it is closed. And transaction.manager / pyramid_tm is making each session garbage collected after each request, isn't it? Thus add_finished_callback is implemented in SQLAlchemy side, at least this is

Re: [pylons-discuss] Pyramid + SQLAlchemy + PostgreSQL: idle connections

> On Apr 2, 2018, at 09:54, Jonathan Vanasco wrote: > > I don't know why it was removed from the cookiecutter. > > It's been present in the cookbook and docs for years, see the cookbook: > >

Re: [pylons-discuss] Pyramid + SQLAlchemy + PostgreSQL: idle connections

On Sunday, April 1, 2018 at 8:18:39 PM UTC-4, Zsolt Ero wrote: > > Hi Jonathan, > > I'm not 100% sure I understand when are you talking about the > "standard" Pyramid way of the cookiecutter template's get_tm_session's > implementation and when are you talking about my special addition of >