[sqlalchemy] Sessions, threads, cherrypy visibility

2012-01-17 Thread John Hufnagle
Didn't see an answer to this in the discussions.
I have:
1. slqlchemy 0.7.3, cherrypy and mysql  Ver 14.14 Distrib 5.5.18, for
osx10.6

2. a cherry py application that initializes the scoped session as:
Session = scoped_session(sessionmaker(bind=engine))

3a. Passes that Session instance into a background worker thread.
3b. Passes that Session instance into the Request processing objects
that handle the http requests.

4. in the run() method of that background thread (3a) it issues:
session_obj = passed_in_session_object()  # passed_in_session_object
came in from the main from step 3a.
#this session_obj is used for SELECT's against the db
wakes up every minute and does SELECTS to see if the http handler from
5 (below) has written any objects to the db.

5. in the request processing code that handles the http requests it
also creates a thread specific session object similar to step 4 above
session_obj = passed_in_session_object()
performs CREATE's of db rows that should be seen by the background
worker thread in step 4

The trouble is the records get written to the db but the SELECT stmts
that run on background thread in #4 get 0 results for records that are
in the db.
However if for test purposes I issue a SELECT in the http handling
code after the object has been CREATE'd and the session has been
commit'ed I can see the object.

Am I doing this wrong?

Thanks
John

-- 
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to 
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en.



Re: [sqlalchemy] Sessions, threads, cherrypy visibility

2012-01-17 Thread Michael Bayer

On Jan 17, 2012, at 9:03 AM, John Hufnagle wrote:

 Didn't see an answer to this in the discussions.
 I have:
 1. slqlchemy 0.7.3, cherrypy and mysql  Ver 14.14 Distrib 5.5.18, for
 osx10.6
 
 2. a cherry py application that initializes the scoped session as:
 Session = scoped_session(sessionmaker(bind=engine))
 
 3a. Passes that Session instance into a background worker thread.
 3b. Passes that Session instance into the Request processing objects
 that handle the http requests.


right off, I think it's important here to understand what scoped_session means. 
  The scoped_session is only a *registry* that stores sessions, it's not a 
Session object itself.   When you first call upon it, that is, Session(), 
*then* you get at an actual Session object, keyed to the current thread.   If 
you call methods on the scoped_session object itself, those are proxied into 
the Session that's local tot he current thread.

 
 4. in the run() method of that background thread (3a) it issues:
 session_obj = passed_in_session_object()  # passed_in_session_object
 came in from the main from step 3a.
 #this session_obj is used for SELECT's against the db
 wakes up every minute and does SELECTS to see if the http handler from
 5 (below) has written any objects to the db.

So it's not clear here at what point you deal with Session and at what point 
you deal with the actual Session object returned by Session().   If you're 
passing actual Session objects between threads (which is not really recommended 
unless you're really sure of what you're doing) you'd need to make sure its the 
Session object, not the scoped_session registry.

Overall I would refrain from passing Session anywhere, the recommended usage 
of scoped_session is as a module level global that's accessed as needed in a 
thread-local fashion.  I'm not sure what your gaining by passing sessions 
between threads.   Each Session has its own transaction going on, and as long 
as those transactions remain open they won't see any changes that have occurred 
since the transaction is opened (note that transactions are typically isolated 
from seeing concurrent changes elsewhere).

It's not like you can't pass sessions between threads and such but that's an 
advanced technique where you'd need to be very aware of which sessions have 
what transactional state going on.  Hence the scoped_session as a global, 
threadlocal registry, where all threads consult it, open a session, do their 
work, then close it, and that's it, is the default model, being the easiest 
to understand.


-- 
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to 
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en.