the message you're getting is due to an exception being raised,but rollback() not being called on the session. When you catch those ConcurrentModification exceptions, you have to issue a rollback().
I dont see anything else with the code that would suggest the same row being deleted in two places, although it does seem like theres likely a more succinct way of structuring that code, and I also dont see the purpose that the session.expire_all() accomplishes. Things missing here include what scope is the session managed under (i.e. where is it created/destroyed/shared among threads) as well as what other concurrency exists within this section of the application. On Oct 27, 2008, at 9:15 PM, Doug Farrell wrote: > > Hi all, > > I'm using Python 2.5.1 with SqlAlchemy 0.5rc2 with Sqlite on Windows > Server 2003 and I'm having a problem with my application throwing > InvalidRequestError and ConcurrentModification exceptions. Here is my > simplified declarative class: > > class Job(srsmanagerdb.Base): > STATUS_INIT = 0 > STATUS_RUN = 1 > STATUS_DONE = 2 > STATUS_FAIL = 3 > __tablename__ = "jobs" > id = Column(Integer, primary_key=True, > autoincrement=True) > nas = Column(String(12), default=None) > filename = Column(String(64), default=None, index=True) > filesize = Column(Integer, default=None) > created = Column(DateTime, default=None) > job_id = Column(String(32), default=None) > productType = Column(String(1), default=None) > contentType = Column(String(10), default=None) > priorityType = Column(String(10), default=None) > priority = Column(Integer, default=None) > assignedPress = Column(Integer, default=None) > status = Column(Integer, default=None) > > def __init__(self, objrefs, fileDetails): > nas, filename, filesize, ctime = fileDetails > self.nas = nas > self.filename = filename > self.filesize = filesize > self.created = > datetime.datetime(*time.strptime(ctime[:ctime.find(".")], "%Y-%m-%d > %H:%M:%S")[0:6]) > > This object is used to track state information about jobs being > handled > by a looping state machine. I keep a list of all active jobs in a Jobs > collection class, so there are many active intances of the above > class. > The simplified Jobs collection class looks like this: > > class Jobs(AppContext): > def __init__(self, objrefs): > self._logger = __logger__ > self._jobs = [] > self._markedForRemoval = [] > def markForRemoval(self, job): > self._markedForRemoval.append(job) > def removeMarkedJobs(self): # throws exception in > here > session = srsmanagerdb.Session() > for markedJob in self._markedForRemoval: > try: > session.expire_all() > session.delete(markedJob) > session.commit() > self._jobs.remove(markedJob) > except sqlalchemy.exceptions.ConcurrentModificationError, > e: > self._logger.warn("%s threw exception %s" % > (job.filename, e)) > self._markedForRemoval = [] > def process(self): > for job for self._jobs: > job.process() > if job.status == Job.STATUS_DONE: > self.markForRemoval(job) > self.removeMarkedJobs() > > The above simplified code runs for awhile (10s of minutes) with > hundreds > of jobs and then it throws the exception below in the > removeMarkedJobs() > method. I've worked really hard trying to figure out what's going > wrong > here. This is the only place where I delete jobs and commit that > delete > to the database. One question I have is if it's a good idea to keep a > list of active Job instances (database rows) in a Python list. In my > removeMarkedJobs() I'm deleting the job instances, and then removing > the > job instance from the list. Is this necessary or good practice? I > haven't figured out if just deleting the job instance from the list > (self._jobs.remove(markedJob)) will also delete the job from the > database or not. Anyway, here's the traceback of the exception I'm > getting. Any help would be appreciated. > > Thanks, Doug > > 2008-10-27 18:15:54 srsmanager ERROR unexpected error, > restarting: > Traceback (most recent call last): > File "c:\cygwin\home\ripadmin\dev\srsmanager\srsprocess.py", line 154, > in runjobs isActive = self._jobs.process() > File "c:\cygwin\home\ripadmin\dev\srsmanager\jobs.py", line 436, in > process self.removeMarkedJobs() > File "c:\cygwin\home\ripadmin\dev\srsmanager\jobs.py", line 397, in > removeMarkedJobs self._logger.warn("%s threw exception %s" % > (markedJob.filename, e)) > File > "c:\python\2.5\lib\site-packages\SQLAlchemy-0.5.0rc1-py2.5.egg > \sqlalchem > y\orm\attributes.py", line 135, in __get__ return > self.impl.get(instance_state(instance)) > File > "c:\python\2.5\lib\site-packages\SQLAlchemy-0.5.0rc1-py2.5.egg > \sqlalchem > y\orm\attributes.py", line 327, in get value = callable_() > File > "c:\python\2.5\lib\site-packages\SQLAlchemy-0.5.0rc1-py2.5.egg > \sqlalchem > y\orm\attributes.py", line 909, in __call__ attr.impl.key in > unmodified > File > "c:\python\2.5\lib\site-packages\SQLAlchemy-0.5.0rc1-py2.5.egg > \sqlalchem > y\orm\mapper.py", line 1715, in _load_scalar_attributes result = > session.query(mapper)._get(identity_key, refresh_state=state, > only_load_props=attribute_names) > File > "c:\python\2.5\lib\site-packages\SQLAlchemy-0.5.0rc1-py2.5.egg > \sqlalchem > y\orm\query.py", line 1211, in _get return q.all()[0] > File > "c:\python\2.5\lib\site-packages\SQLAlchemy-0.5.0rc1-py2.5.egg > \sqlalchem > y\orm\query.py", line 985, in all return list(self) > File > "c:\python\2.5\lib\site-packages\SQLAlchemy-0.5.0rc1-py2.5.egg > \sqlalchem > y\orm\query.py", line 1073, in __iter__ return > self._execute_and_instances(context) > File > "c:\python\2.5\lib\site-packages\SQLAlchemy-0.5.0rc1-py2.5.egg > \sqlalchem > y\orm\query.py", line 1076, in _execute_and_instances result = > self.session.execute(querycontext.statement, params=self._params, > mapper=self._mapper_zero_or_none(), _state=self._refresh_state) > File > "c:\python\2.5\lib\site-packages\SQLAlchemy-0.5.0rc1-py2.5.egg > \sqlalchem > y\orm\session.py", line 750, in execute return > self.__connection(engine, close_with_result=True).execute( File > "c:\python\2.5\lib\site-packages\SQLAlchemy-0.5.0rc1-py2.5.egg > \sqlalchem > y\orm\session.py", line 717, in __connection return > self.transaction._connection_for_bind(engine) > File > "c:\python\2.5\lib\site-packages\SQLAlchemy-0.5.0rc1-py2.5.egg > \sqlalchem > y\orm\session.py", line 309, in _connection_for_bind > self._assert_is_active() > File > "c:\python\2.5\lib\site-packages\SQLAlchemy-0.5.0rc1-py2.5.egg > \sqlalchem > y\orm\session.py", line 244, in _assert_is_active "The transaction > is inactive due to a rollback in a " InvalidRequestError: The > transaction is inactive due to a rollback in a subtransaction. Issue > rollback() to cancel the transaction > > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~----------~----~----~----~------~----~------~--~---