On Sun, Jun 15, 2014 at 01:26:47PM -0700, Jonathan Vanasco wrote: > i'm using sqlalchemy in pyramid, under uwsgi. > > i've been reading up on forking issues that can arise in this stack. > > from what I understand, issues arise because the "SqlAlchemy Connection" is > established before the fork, resulting in each forked process potentially > trying to use the same connection. One of the fixes is to (re)connect > after the fork. > > This is where i'm confused... I don't know what a "SqlAlchemy Connection" > refers to. > > I've read up on the uwsgi, sqlalchmy and pylons lists + docs -- I can't get > any better info on what this SqlAlchemy connection is, or what component is > having the issues. > > If the issue is with the SqlAlchemy engine connecting to the database, and > it's internal db connection, then I should be fine -- SqlAlchemy doesn't > connect in my apps until the first request. I verified this with > postgresql logs. > > If the issue is with creating a SqlAlchemy engine or sessionmaker, then I > will need to rewrite some stuff. > > Does anyone know the specifics of the "connection" that has fork issues? >
A problem that can arise when using sqla connection pooling is (I'm not sure that this happens under all conditions, but it has happened to me): 1. In the master process (e.g. in pyramid main()), engine is set up (with pooling) a sqlconnection made, used, and closed. The connection, in this case is not actually closed, but just returned to the pool. 2. Fork (e.g. uwsgi forks the workers) 3. Each child process thinks it's got a connection in its pool. The problem is it's the same connection in each process. When multiple process try to use the same connection, it's bad, obviously. One fix, if you make connections in the master process, is, instead of closing the connection (connection.close()) invalidate it (connection.invalidate()). Then the underlying connection is actually closed, rather than returned to the pool. There some on this in this thread: https://groups.google.com/forum/#!searchin/sqlalchemy/UnicodeEncodeErrors/sqlalchemy/Xf0fWsCqdCg/5-6aRpYcuecJ Jeff -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.
