[sqlalchemy] Re: Can anyone explain me why my thread block ???
Hi,
I found what block my print in my thread.
It s not sql alchemy library, it s MySQLdb...
They import directly in the function, import block print in threads
I modified the library (imports move out of the method) and it works
very well now. I can now run asynchronous queries so my gui don t freeze.
from constants import CLIENT, FIELD_TYPE
from converters import conversions
from weakref import proxy, WeakValueDictionary
import types
kwargs2 = kwargs.copy()
if kwargs.has_key('conv'):
conv = kwargs['conv']
else:
--
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalch...@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] No import in function and method = thread firendly
Hi again, I understand all! SqlAlchemy and MySQLdb don t respect this rule 16.2.9 http://docs.python.org/library/threading.html#threaded-imports Why? :( -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] No import in function and method = thread firendly
Where do you see SQLAlchemy spawning any threads ?Its the responsibility of the threaded application to make sure all imports occur before threads are spawned. The documentation you refer to states as much. On May 2, 2010, at 8:34 AM, christiandemo...@gmail.com wrote: Hi again, I understand all! SqlAlchemy and MySQLdb don t respect this rule 16.2.9 http://docs.python.org/library/threading.html#threaded-imports Why? :( -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] No import in function and method = thread firendly
correction, it doesn't even say that. All it says regarding deadlocks is that an import shouldn't spawn a thread. SQLAlchemy does not spawn any threads (pretty sure MySQLdb doesn't, either). On May 2, 2010, at 11:28 AM, Michael Bayer wrote: Where do you see SQLAlchemy spawning any threads ?Its the responsibility of the threaded application to make sure all imports occur before threads are spawned. The documentation you refer to states as much. On May 2, 2010, at 8:34 AM, christiandemo...@gmail.com wrote: Hi again, I understand all! SqlAlchemy and MySQLdb don t respect this rule 16.2.9 http://docs.python.org/library/threading.html#threaded-imports Why? :( -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] No import in function and method = thread firendly
Hello Michael, Thx for answer. Of course, there is no thread in mysqldb but there is import in function connect line 73 of __init__.py and line 128 to 132 of connections.py in method __init__(self, *args, **kwargs): So when i use connect function in my thread, it blocks because it encounters the import of the library. I rework mysqldb files, i move import out of functions and my thread program which use mysqldb works : no deadlock. I tried to do the same with sqlalchemy library but it s too hard, i failed. So i can only use mysqldb in my thread though i want to use sqlalchemy which is better to manipulate db. correction, it doesn't even say that. All it says regarding deadlocks is that an import shouldn't spawn a thread. SQLAlchemy does not spawn any threads (pretty sure MySQLdb doesn't, either). On May 2, 2010, at 11:28 AM, Michael Bayer wrote: Where do you see SQLAlchemy spawning any threads ?Its the responsibility of the threaded application to make sure all imports occur before threads are spawned. The documentation you refer to states as much. On May 2, 2010, at 8:34 AM, christiandemo...@gmail.com wrote: Hi again, I understand all! SqlAlchemy and MySQLdb don t respect this rule 16.2.9 http://docs.python.org/library/threading.html#threaded-imports Why? :( -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] No import in function and method = thread firendly
Hi All it says regarding deadlocks is that an import shouldn't spawn a thread Can you explain me why import math (whick don t spawn a thread) block this thread (only one ok appear in terminal)? When i remove import math, ok appears continuously in term. I don t understand class SyncTopAppel(threading.Thread): def __init__(self): threading.Thread.__init__(self) self.running = False def run(self): self.running = True while self.running: print ok import math time.sleep(1) def stop(self): self.running = False All it says regarding deadlocks is that an import shouldn't spawn a threa -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] No import in function and method = thread firendly
Ok i found this subject which explain why. http://www.mail-archive.com/python-...@python.org/msg24747.html The deadlock happens because strptime has an import inside it, and recursive imports are not allowed in different threads. This situation is well known, found a lot of references to this import-thread-import problem in discussions and previous bugs (i.e.: http://bugs.python.org/issue683658). I'm not sure how anyone could tell more. There are some locks and you're staring at a deadlock. This isn't rare in threaded programs, alas. Are you by any chance spawning and executing a thread as a side-effect of doing an import, where the spawned thread itself does an import and the spawning code waits for the spawned thread to finish? If so, the spawning thread would wait forever on whatever gimmick it's using to wait for the spawned thread to finish, and the spawned thread would wait forever for the global import lock to get released (which can't happen, because the global import lock is waiting for the spawning thread to release it, but the spawning thread is waiting for the spawned thread to finish). Or, more simply, never spawn a thread as a side-effect of importing. Sry for inconvenience Chris correction, it doesn't even say that. All it says regarding deadlocks is that an import shouldn't spawn a thread. SQLAlchemy does not spawn any threads (pretty sure MySQLdb doesn't, either). On May 2, 2010, at 11:28 AM, Michael Bayer wrote: Where do you see SQLAlchemy spawning any threads ?Its the responsibility of the threaded application to make sure all imports occur before threads are spawned. The documentation you refer to states as much. On May 2, 2010, at 8:34 AM, christiandemo...@gmail.com wrote: Hi again, I understand all! SqlAlchemy and MySQLdb don t respect this rule 16.2.9 http://docs.python.org/library/threading.html#threaded-imports Why? :( -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] session lifecycle and wsgi
On Apr 29, 2010, at 3:21 AM, Chris Withers wrote: Michael Bayer wrote: if your application keeps a handle on objects after the request is complete, and then passed them somewhere else, like a background thread or something, then the subsequent request is going to be potentially touching those objects at the same time. This would all be pretty poor practice as individual threads should always have their own sessions. Right, you should either be .close() or .remove()'ing the session here or manually expunging the objects you want to shift to the other thread, correct? (I'm guessing session.merge will whine if handed an object that is already in another session?) mm no merge() leaves the original unaffected. it copies state to an instance internal to the session. this is very clear here: http://www.sqlalchemy.org/docs/session.html#merging Or maybe you loaded those objects into a globally-scoped in-memory cache of some kind - you probably don't want the next request touching them directly as once they're in a global cache of some kind, other threads would be calling upon them to copy their state locally. So putting non-expunged objects in something like a beaker cache would be a no-no, correct? (would .close() or .remove() fix the problem if the objects are already in the cache by the time the .close() or .remove() is called?) in most cases its actually fine.file, memcached, reldb, dbm backends all serialize the given object, which means you're only storing a copy.If the cache is storing things locally to the current session (see examples/beaker_cache/local_session_caching.py), then you dont want to expunge the object since you'd like it to be in the session at the same time. only in-memory, non-session-scoped caches have this limitation, such as if you were using a memory backend with beaker. On the other hand, you might load the objects into a session-local cache of some kind that you've created.Now, when the next request comes in and calls upon those same rows, the ORM doesn't need to re-instantiate the objects, they are already present in the cache. Does the ORM check if the attributes of the cached object are correct or would you end up in a situation where you do a query but end up using the cached attributes rather than the ones just returned from the db? that all depends how you get the object from a cache back into your session. usually not since having to hit the DB to verify attributes defeats the purpose of a cache. pretty much only if you used merge() with load=True. This is a use case where you'd want to keep the same session from one request to the next. It looks like this is what zope.sqlalchemy does. The last thing it does in a logical transaction (ie: a http://pypi.python.org/pypi/transaction transaction) is call .close(). Should it be calling .remove()? http://www.sqlalchemy.org/docs/session.html#lifespan-of-a-contextual-session would seem to imply that the .close() alone is fine? close() alone should be fine. the bulletpoints at http://www.sqlalchemy.org/docs/session.html#lifespan-of-a-contextual-session are the best I can do here, I'm really just repeating myself over and over in these threads Well, except the info you've given in this thread, at least, goes way above what's in that list ;-) A few comments on the bullets: - They don't explain what happens to transactions and connections. The points for both remove() and close() say all of its transactional/connection resources are closed out; does this mean database connections or closed or just returned to the pool? (I hope the latter!) closed out means rolled back and returned to thoe pool. - The point for .commit() states The full state of the session is expired, so that when the next web request is started, all data will be reloaded but your last reply implied this wouldn't always be the case. The instantiated objects that are in the session still stay around as long as they are referenced externally. but all their attributes are gone, as well as the new and deleted collections are empty. so all data will be reloaded. Also, is it fair game to assume that session.close() rolls back any open database transaction? Is there any difference between that rollback and calling session.rollback() explicitly? i think the rollback which close() might get to the point more directly internallybut from a connection point of view there's no different. Finally, in nosing around session.py, I notice that SessionTransactions can be used as context managers. Where can I find good examples of this? you'd be saying with session.begin(): -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For
