[sqlalchemy] Re: Concurrency problem with SA 0.4
On Oct 22, 2007, at 4:25 AM, alain D. wrote: Obj is a simple python object with on field (key : String(10)) mapped to a table with on field (key) with is the primary key. Session is defined like this : engine = create_engine(**mysql_string**, strategy='threadlocal', pool_size=50) Session = scoped_session(sessionmaker(bind=engine, autoflush=True, transactional=True)) If I launch 2 python shells : In the first I do : sess = Session() obj = Obj(first_key) sess.save() sess.commit() In the second I do sess = Session() #this is done at the same time than the creation in the first shell # wait after commit in shell 1 oo = sess.query(Obj).filter_by(key=first_key).first # oo is None its likely that a transaction is begun in the second session before the first session commits - transaction isolation will prevent #2 from seeing new changes until its either rolled back or committed. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Concurrency problem with SA 0.4
I forgot to say that I use mysql with InnoDB backend (autoflush=True,transactional=True) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Concurrency problem with SA 0.4
On Oct 22, 2007, at 1:43 PM, alain D. wrote: thanks a lot for you reply ... If I understand correctly, a simple sess.begin() in #2 executed right after the #1 commit will work (?). But I tested it and I still dont get the commited object ... What am I missing no, a sess.rollback() or sess.commit() in #2. sess.begin() wont do anything here. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Concurrency problem
On Apr 2, 2007, at 10:35 PM, Barry Warsaw wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Our populate_instance() checks the isnew flag and if this is true, it does not call our initialization stuff. Now, I don't really see the isnew flag documented, so I assumed it was only called when the object was initially loaded from the database, but apparently it's true /every/ time the object is loaded. So when I session.expire() it and then subsequently load it, populate_instance() gets called with isnew=True. The causes our application-specific initialization stuff to get called on an existing MailList object, and this is what blows away our lock attribute. I guess that if the documentation for isnew in append_result() is the same as isnew in populate_instance() then indeed we'll need that extra check. Is that correct? OK, yes, im sorry about the lack of docs for isnew...sometimes i subconsciously want to see how long it takes for someone to ask me about something (bad habit) although in this case that flag is over a year old and i think youre the first :) isnew is used to indicate that we are going to populate the instance with data from the database, *and* that this particular row is the first row in the result which has indicated the presence of this entity (i.e. the primary key points to it). this implies that populate_instance() can be called *multiple times* for the instance, if multiple successive rows all contain its particular primary key. this is how SA deals with the products of eager loads and other joins, where many rows contain the same entity identity in a number of successive rows corresponding to the total product of eagerly loaded child items - the normal populate_instance() just delegates the populate this attribute task onto MapperProperty objects that represent each attribute. for the MapperProperty objects that represent eager loaders, they unconditionally send the row off to their child mappers to look at the row, where it then goes into its own populate_instance() process with a possibly True isnew flag. so basically, isnew means populate attributes and if its false it means we already did this entity. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Concurrency problem
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Apr 3, 2007, at 9:35 AM, Michael Bayer wrote: OK, yes, im sorry about the lack of docs for isnew...sometimes i subconsciously want to see how long it takes for someone to ask me about something (bad habit) although in this case that flag is over a year old and i think youre the first :) Heh, um glad to help? :) so basically, isnew means populate attributes and if its false it means we already did this entity. Cool, thanks. That tells me what I need to know! - -Barry -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRhKhIXEjvBPtnXfVAQIJAAP/XRTViMrga4tOECEJD0OkacjfcUCVQOV3 lDUTPtu09se0uiPlZvfW03w7BBniguupLFSUOPBQHlf/Zh3SWG585lF9HeNP0ocC viBepxS+2rm1Y3CZVnXvfNwZYUfpn1vwns2+cOFoHnwR8scdexK9UTAG7yIH/ICs IzNwbKB/g7U= =bDwL -END PGP SIGNATURE- --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Concurrency problem
On Apr 2, 2007, at 1:46 PM, Barry Warsaw wrote: The problem is that if we call session.expire(mlist) /after/ the MailList.__lock attribute gets set, it appears as though SA 'messes' with this attribute. I've proven that if the session.expire() gets called after MailList.Lock(), during the .Load() we lose our lock, but if we call session.expire() before we're locked, everything's fine. I'm not 100% sure yet why that happens, but it does appear to be a strange interaction between SA's identity map and non-SA controlled attributes. I cant think of any reason why SA would be touching object attributes which are not part of a mapping. can you be more specific about how this object is mapped and in what way is it related to other objects ? (i.e. are you sure its the *same* instance of MailList ? thats the only reason i could think that its a different __lock attribute). also, why dont you just use the load() method on Query to load the object ? this will overwrite all attributes of the instance with that loaded from the database; expire() would not be needed. So, I'm looking for some advice on how to handle synchronization between processes in a situation like this. Ultimately I think I'd like to use mapper's always_refresh, but maybe that's not the right thing. the always_refresh flag was added at the extremely vocal requests of a single user who refused to accept the concepts of identity map and unit of work. I cant make any recommendations how such a flag should be used. the load() operation should be a more controllable way to handle loading. The related issue is that MailList's may have other ORM objects hanging off of it and they'd all need to be re-sync'd with the database at the same time. Our MailList objects hang around for a long time, and so we already have the resync locations coded (essentially MailList.Load() calls). Those are the points where we'd like to expire our in-memory objects, though we can't lose the state of our non-SA controlled attributes. heres the big catch - the objects that you place in your Session are generally not going to respond well to concurrent access, since the Session itself is not threadsafe nor are the attribute-level operations performed on instances. You might want to look into separating the database-related state of MailList into an entity that is useable on a more ad-hoc basis, so that MailList, its lock attributes, and its synchronization needs arent impacted by SA session semantics which are intended to be single-thread operations. alternatively, you can try having your long-lived MailList living outside of the session, and to merge() it into a session when you need to do flushes (merge does not impact the original instance, but rather creates a copy of the instance within the session). the feeling im getting here, without seeing how you have things set up, is that youre hanging a lot of functionality off a single type of object (locking, persistence, etc), and perhaps this is where a greater notion of separation of concerns might be helpful. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
