Michael, I'm not sure why the ConcurrentModification is occurring at all, my application doesn't use threads. I'll look into issuing the rollback() call and see how that helps my app.
I'm sure there are better ways to structure my code and use of SA, this is the result of prototyping along in an effort to learn, understand and use SA. The session.expire_all() was injected recently as the result of reading some stuff during a Google Search and just casting about to solve the thrown exceptions. I also had a job = session.merge(job) in there at one time. I have one module called managerdb.py where I create the engine and the first session object. All other modules reference this module to create their own session objects. By the way, with locally scoped session variables should I do a session.close() or not? Thanks, Doug > -----Original Message----- > From: sqlalchemy@googlegroups.com [mailto:[EMAIL PROTECTED] > On Behalf Of Michael Bayer > Sent: Tuesday, October 28, 2008 10:20 AM > To: sqlalchemy@googlegroups.com > Subject: [sqlalchemy] Re: InvalidRequestError and > ConcurrentModification problems > > > the message you're getting is due to an exception being raised,but > rollback() not being called on the session. When you catch those > ConcurrentModification exceptions, you have to issue a rollback(). > > I dont see anything else with the code that would suggest the same row > being deleted in two places, although it does seem like theres likely > a more succinct way of structuring that code, and I also dont see the > purpose that the session.expire_all() accomplishes. Things missing > here include what scope is the session managed under (i.e. where is it > created/destroyed/shared among threads) as well as what other > concurrency exists within this section of the application. > > On Oct 27, 2008, at 9:15 PM, Doug Farrell wrote: > > > > > Hi all, > > > > I'm using Python 2.5.1 with SqlAlchemy 0.5rc2 with Sqlite on Windows > > Server 2003 and I'm having a problem with my application throwing > > InvalidRequestError and ConcurrentModification exceptions. Here is my > > simplified declarative class: > > > > class Job(srsmanagerdb.Base): > > STATUS_INIT = 0 > > STATUS_RUN = 1 > > STATUS_DONE = 2 > > STATUS_FAIL = 3 > > __tablename__ = "jobs" > > id = Column(Integer, primary_key=True, > > autoincrement=True) > > nas = Column(String(12), default=None) > > filename = Column(String(64), default=None, index=True) > > filesize = Column(Integer, default=None) > > created = Column(DateTime, default=None) > > job_id = Column(String(32), default=None) > > productType = Column(String(1), default=None) > > contentType = Column(String(10), default=None) > > priorityType = Column(String(10), default=None) > > priority = Column(Integer, default=None) > > assignedPress = Column(Integer, default=None) > > status = Column(Integer, default=None) > > > > def __init__(self, objrefs, fileDetails): > > nas, filename, filesize, ctime = fileDetails > > self.nas = nas > > self.filename = filename > > self.filesize = filesize > > self.created = > > datetime.datetime(*time.strptime(ctime[:ctime.find(".")], "%Y-%m-%d > > %H:%M:%S")[0:6]) > > > > This object is used to track state information about jobs being > > handled > > by a looping state machine. I keep a list of all active jobs in a > Jobs > > collection class, so there are many active intances of the above > > class. > > The simplified Jobs collection class looks like this: > > > > class Jobs(AppContext): > > def __init__(self, objrefs): > > self._logger = __logger__ > > self._jobs = [] > > self._markedForRemoval = [] > > def markForRemoval(self, job): > > self._markedForRemoval.append(job) > > def removeMarkedJobs(self): # throws exception in > > here > > session = srsmanagerdb.Session() > > for markedJob in self._markedForRemoval: > > try: > > session.expire_all() > > session.delete(markedJob) > > session.commit() > > self._jobs.remove(markedJob) > > except sqlalchemy.exceptions.ConcurrentModificationError, > > e: > > self._logger.warn("%s threw exception %s" % > > (job.filename, e)) > > self._markedForRemoval = [] > > def process(self): > > for job for self._jobs: > > job.process() > > if job.status == Job.STATUS_DONE: > > self.markForRemoval(job) > > self.removeMarkedJobs() > > > > The above simplified code runs for awhile (10s of minutes) with > > hundreds > > of jobs and then it throws the exception below in the > > removeMarkedJobs() > > method. I've worked really hard trying to figure out what's going > > wrong > > here. This is the only place where I delete jobs and commit that > > delete > > to the database. One question I have is if it's a good idea to keep a > > list of active Job instances (database rows) in a Python list. In my > > removeMarkedJobs() I'm deleting the job instances, and then removing > > the > > job instance from the list. Is this necessary or good practice? I > > haven't figured out if just deleting the job instance from the list > > (self._jobs.remove(markedJob)) will also delete the job from the > > database or not. Anyway, here's the traceback of the exception I'm > > getting. Any help would be appreciated. > > > > Thanks, Doug > > > > 2008-10-27 18:15:54 srsmanager ERROR unexpected error, > > restarting: > > Traceback (most recent call last): > > File "c:\cygwin\home\ripadmin\dev\srsmanager\srsprocess.py", line > 154, > > in runjobs isActive = self._jobs.process() > > File "c:\cygwin\home\ripadmin\dev\srsmanager\jobs.py", line 436, in > > process self.removeMarkedJobs() > > File "c:\cygwin\home\ripadmin\dev\srsmanager\jobs.py", line 397, in > > removeMarkedJobs self._logger.warn("%s threw exception %s" % > > (markedJob.filename, e)) > > File > > "c:\python\2.5\lib\site-packages\SQLAlchemy-0.5.0rc1-py2.5.egg > > \sqlalchem > > y\orm\attributes.py", line 135, in __get__ return > > self.impl.get(instance_state(instance)) > > File > > "c:\python\2.5\lib\site-packages\SQLAlchemy-0.5.0rc1-py2.5.egg > > \sqlalchem > > y\orm\attributes.py", line 327, in get value = callable_() > > File > > "c:\python\2.5\lib\site-packages\SQLAlchemy-0.5.0rc1-py2.5.egg > > \sqlalchem > > y\orm\attributes.py", line 909, in __call__ attr.impl.key in > > unmodified > > File > > "c:\python\2.5\lib\site-packages\SQLAlchemy-0.5.0rc1-py2.5.egg > > \sqlalchem > > y\orm\mapper.py", line 1715, in _load_scalar_attributes result = > > session.query(mapper)._get(identity_key, refresh_state=state, > > only_load_props=attribute_names) > > File > > "c:\python\2.5\lib\site-packages\SQLAlchemy-0.5.0rc1-py2.5.egg > > \sqlalchem > > y\orm\query.py", line 1211, in _get return q.all()[0] > > File > > "c:\python\2.5\lib\site-packages\SQLAlchemy-0.5.0rc1-py2.5.egg > > \sqlalchem > > y\orm\query.py", line 985, in all return list(self) > > File > > "c:\python\2.5\lib\site-packages\SQLAlchemy-0.5.0rc1-py2.5.egg > > \sqlalchem > > y\orm\query.py", line 1073, in __iter__ return > > self._execute_and_instances(context) > > File > > "c:\python\2.5\lib\site-packages\SQLAlchemy-0.5.0rc1-py2.5.egg > > \sqlalchem > > y\orm\query.py", line 1076, in _execute_and_instances result = > > self.session.execute(querycontext.statement, params=self._params, > > mapper=self._mapper_zero_or_none(), _state=self._refresh_state) > > File > > "c:\python\2.5\lib\site-packages\SQLAlchemy-0.5.0rc1-py2.5.egg > > \sqlalchem > > y\orm\session.py", line 750, in execute return > > self.__connection(engine, close_with_result=True).execute( File > > "c:\python\2.5\lib\site-packages\SQLAlchemy-0.5.0rc1-py2.5.egg > > \sqlalchem > > y\orm\session.py", line 717, in __connection return > > self.transaction._connection_for_bind(engine) > > File > > "c:\python\2.5\lib\site-packages\SQLAlchemy-0.5.0rc1-py2.5.egg > > \sqlalchem > > y\orm\session.py", line 309, in _connection_for_bind > > self._assert_is_active() > > File > > "c:\python\2.5\lib\site-packages\SQLAlchemy-0.5.0rc1-py2.5.egg > > \sqlalchem > > y\orm\session.py", line 244, in _assert_is_active "The > transaction > > is inactive due to a rollback in a " InvalidRequestError: The > > transaction is inactive due to a rollback in a subtransaction. Issue > > rollback() to cancel the transaction > > > > > > > > > > t this group at http://groups.google.com/group/sqlalchemy?hl=en > -~----------~----~----~----~------~----~------~--~--- --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~----------~----~----~----~------~----~------~--~---
