Re: [sqlalchemy] Re: using scoped_session in serial mode as well as parallel (multi-threaded) mode
On Aug 12, 2010, at 9:25 AM, Faheem Mitha wrote: Hi Mike, Thanks for the response, but I don't follow. When you say multiple threads are hitting the session you have above, which session are you referring to? There is more than one object above that could be called a session. Ie. I don't actually know, it would require that I have a full install of your application for me to run and step through with a debugger to fully understand what it's doing. All I can say from here is that the errors you have suggest concurrent access to a single psycopg2 connection resource, and that a single Session references a single connection when in use. A MetaData object does not, nor does an Engine - only a Session.If you remove all threading from your application and the errors go away, then you know you're accessing some resource illegally. Session1 in Session1 = sessionmaker() session1 in session1 = Session1(bind=db) Session2 in Session2 = scoped_session(sessionmaker()) Let me try to ask a precise question. If I do Session = scoped_session(sessionmaker()) then is it ok for this Session object to be be passed around between multiple threads and used directly as in Session.commit() Does this correspond to a single psycopyg2 connection? If it does, and this usage is wrong, should I be creating separate sessions within each thread like That's a scoped_session, which is threadsafe. Everything you call upon it will acquire a Session object from a thread local context, and commit() is called on that (for information on thread locals, see http://docs.python.org/library/threading.html#threading.local. If you pass a scoped_session from one thread to another, and the second thread calls commit(), the second thread is not affecting the transaction begun by the first. They are two separate transactions. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Re: using scoped_session in serial mode as well as parallel (multi-threaded) mode
On Aug 12, 2010, at 11:47 AM, Faheem Mitha wrote: On Thu, 12 Aug 2010 09:44:04 -0400, Michael Bayer mike...@zzzcomputing.com wrote: On Aug 12, 2010, at 9:25 AM, Faheem Mitha wrote: Hi Mike, Thanks for the response, but I don't follow. When you say multiple threads are hitting the session you have above, which session are you referring to? There is more than one object above that could be called a session. Ie. I don't actually know, it would require that I have a full install of your application for me to run and step through with a debugger to fully understand what it's doing. All I can say from here is that the errors you have suggest concurrent access to a single psycopg2 connection resource, and that a single Session references a single connection when in use. A MetaData object does not, nor does an Engine - only a Session. If you remove all threading from your application and the errors go away, then you know you're accessing some resource illegally. Yes, I see. Yes, the error does not show up unless I run multiple threads, and I agree with your interpretation. If MetaData is threadsafe, then using ThreadLocalMetaData is not necessary? ThreadLocalMetaData is not necessary and is not used for this purpose. Ok. Thanks for the confirmation. So, if I was to use scoped sessions systematically everywhere, this problem would likely disappear. that's not necessarily true - if you share individual persistent objects among threads, they reference their owning session in order to load additional state. If you pass objects between threads you should merge() them into the current thread's session first, then use that result. Can you confirm that there is no reason not to use scoped sessions everywhere, even in serial execution? Of course, if that is the case, then I wonder why non-scoped sessions are used at all. scoped_sessions are usually the default choice for web applications since they desire distinct transactions and object state for individual threads.They are overkill and potentially confusing or inappropriate in other situations, however. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Re: using scoped_session in serial mode as well as parallel (multi-threaded) mode
On Aug 12, 2010, at 2:41 PM, Faheem Mitha wrote: objects among threads, they reference their owning session in order to load additional state. If you pass objects between threads you should merge() them into the current thread's session first, then use that result. I see. That's very enlightening. Can one query such objects to determine their owning session? Some attribute, perhaps? see http://www.sqlalchemy.org/docs/session.html#frequently-asked-questions Can you confirm that there is no reason not to use scoped sessions everywhere, even in serial execution? Of course, if that is the case, then I wonder why non-scoped sessions are used at all. scoped_sessions are usually the default choice for web applications since they desire distinct transactions and object state for individual threads. They are overkill and potentially confusing or inappropriate in other situations, however. I'm not sure why they would be potentially confusing. What are some of the downsides? I'd have thought that not having shared state was less confusing. the scoped_session is a proxy object to the real object. It is less confusing for those unfamiliar with thread locals and proxy objects to deal with a Session directly. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
