[sqlalchemy] Re: Multi-Processing Environment

2011-06-02 Thread ddarko
I use PostgreSQL, but it does not matter, because when someone changes
the configuration it has still work.

a new Engine needs to be created in each child fork

If a worker can perform at any given time only one synchronous request
and for each worker I create a new engine, it does not have any pool.

Do not know if I understand correctly, but it overlooks the fact that
no matter whether it is an entirely new process, or created using fork
() (memory of the parent process is` copied `Effectively)?

The engine is not shared. I need to create it again.

On May 28, 4:53 pm, Michael Bayer mike...@zzzcomputing.com wrote:
 On May 28, 2011, at 4:25 AM, ddarko wrote:









 http://lists.unbit.it/pipermail/uwsgi/2011-May/002078.html:

  ...Unfortunately, when running our
  app in uWSGI with more than one worker, we get sporadic, but frequent,
  sqlalchemy-related exceptions when testing under load. Following is an
  example of one of the more common errors we get.

  Error - class 'sqlalchemy.exc.OperationalError':
  (OperationalError) server closed the connection unexpectedly
         This probably means the server terminated abnormally
         before or while processing the request.

  It would seem that our app, or sqlalchemy, is making an assumption
  that
  is no longer true when running as multiple workers in uWSGI

  uWSGI 0.9.8
  Python 3.2
  SQLAlchemy 0.7.0

  Does anyone use this combination?
  How to share a pool of connections between workers?

 The mechanics of fork() are such that the memory of the parent process is 
 effectively copied to the child process.  The SQLAlchemy Engine by default 
 maintains a pool of database connections, which like everything else is 
 replicated into the child process.  Each database connection in the pool 
 ultimately references a TCP/IP handle to the remote database, unless you're 
 using SQLite in which case it's an open file handle.   These handles lose 
 their meaning when referenced in a new process, that is the child fork, and 
 attempts to use the connections in the new process will fail.

 So the short answer is a new Engine needs to be created in each child fork, 
 or otherwise pooling can be turned off using NullPool.

-- 
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to 
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en.



[sqlalchemy] Re: Multi-Processing Environment

2011-05-28 Thread NiL
are you using sqlite ?
if so :

http://groups.google.com/group/sqlalchemy/browse_thread/thread/5e08e49fa1722f91/55940423d2e6f99f?lnk=gstq=nullpool#55940423d2e6f99f

http://www.sqlalchemy.org/docs/06/core/pooling.html?highlight=nullpool#switching-pool-implementations

might be of help

-- 
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to 
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en.