> -----Original Message----- > From: sqlalchemy@googlegroups.com > [mailto:sqlalch...@googlegroups.com] On Behalf Of gizli > Sent: 02 September 2009 04:45 > To: sqlalchemy > Subject: [sqlalchemy] Curious Problem > > > Hi all, > > I just discovered something weird when doing concurrency testing with > my program. Before writing a simplified test case for it and really > figuring out whether its a bug with sqlalchemy (I am using 0.5.5), I > wanted to write the scenario here. Basically I was getting the > infamous "Set size changed during iteration" error during commit. > Multiple threads are not accessing the same session (as was in some of > the posts I have read online). Here's what I am doing: > > There is a task dispatcher that queries the database, gets a list of > tasks to be done, dispatches them to separate threads. Each thread > creates its own session using the session_maker and updates the status > of a task. There are three kinds of objects in question: Task, > TaskSchedule and TaskResult. There is a one-to-many relation between > Task and TaskSchedule. There is also a one-to-many relation between > Task and TaskResult. Pseudo code: > > task_dispatcher: > for task_schedule in sess.query(TaskSchedule).filter > (next_execution_time <= current_time): > do_task(task_schedule.task) <-- using relation > task_schedule.next_execution_time = some_value > sess.commit() > > (in a new thread) > do_task(task): > sess := get_session() > sess.add(TaskResult(task.id, task_started)) > sess.commit() > task.perform() > sess.merge(TaskResult(task.id, task_finished)) > sess.commit() > sess.close() >
I'm no expert on any of this, but there are a couple of aspects of this that I would imagine could be risky. You are obtaining a 'task' object in your dispatcher thread (so it is attached to that thread's session), and then using it in another thread. If "task.perform" does any ORM-related activity (such as triggering lazy loads), they will be loaded into the dispatcher thread's session, not the subthread's session. Also, if your session has expire_on_commit set (defaults to True when using sessionmaker), then accessing any property at all of the task will trigger a load. I think you should either be expunging your task object from the original session, or at least merging it into your new session. I think you could use session.merge(task, dont_load=True) to prevent the merge from querying the database again. Hope that helps, Simon --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~----------~----~----~----~------~----~------~--~---
