On Thu, 12 Aug 2010 08:47:33 -0400, Michael Bayer <mike...@zzzcomputing.com>
wrote:
>
> On Aug 12, 2010, at 5:10 AM, Faheem Mitha wrote:
>
>>
>> Hi,
>>
>> I'm using scoped_session with PostgreSQL to run multiple threads, each
>> thread creating a table.
> Its generally a poor application practice for an application to need
> new permanent tables on the fly. I think reddit's application might
> do this, but its considered poor design. Database schemas are
> supposed to be fixed.
I'm not sure what you mean by on-the-fly. The app is creating the
tables for later use. It is parallelizing the table creation for
performance reasons - there are a lot of tables.
>> However, I'm getting some pretty weird and inconsistent errors
>> (segfaults from Python with some very peculiar errors), and I was
>> wondering if my usage was at fault.
>> The program has a serial bit, and a parallel bit, and currently I'm doing
>> something like this. For the serial bit I do
>>
>> db = create_engine(dbstring)
>> Session = sessionmaker()
>> session = Session(bind=db)
>>
>> and then later I do
>>
>> session.close()
>>
>> db = create_engine(self.dbstring)
>> Session = scoped_session(sessionmaker())
>> Session.configure(bind=db)
>> Looking at this is seems likely that is would be better to just use
>> scoped_session everywhere, that is, just start with
>> db = create_engine(self.dbstring)
>> Session = scoped_session(sessionmaker())
>> Session.configure(bind=db)
>> [proceed with using Session in serial mode and eventually use it in parallel
>> mode too]
>> I'm basically writing to confirm that it is Ok to use
scoped_session in this way. The way I'm doing it looks a little
dodgy. I don't know if this is really the cause of my problem -
just clutching at straws here. Thanks in advance. Please CC me on
any reply
> you can make as many scoped_sessions, metadatas, etc. as you want,
none of that would cause a segfault. They are just Python
objects. Its only if you share a psycopg2 connection between threads
and have different threads hammer on it simultaneously that there
would be issues like that. If you have a single session, and share
that among threads who access it concurrently, this will produce
that result. There should be no need to guess about it. If
multiple threads are hitting the session you have above, then that's
a likely cause of your issue.
Hi Mike,
Thanks for the response, but I don't follow.
When you say "multiple threads are hitting the session you have
above", which session are you referring to? There is more than one
object above that could be called a session. Ie.
Session1 in "Session1 = sessionmaker()"
session1 in "session1 = Session1(bind=db)"
Session2 in "Session2 = scoped_session(sessionmaker())"
Let me try to ask a precise question. If I do
Session = scoped_session(sessionmaker())
then is it ok for this Session object to be be passed around
between multiple threads and used directly as in
Session.commit()
Does this correspond to a single psycopyg2 connection? If it does, and
this usage is wrong, should I be creating separate sessions within
each thread like
session = Session()
and then doing
session.commit()
within each thread? Or something else? My usage is based on examples
online.
My later posts have more details, along with healthy amounts of
confusion. I apologise for my cluelessness, sorry.
Regards, Faheem
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to sqlalch...@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
