Thanks Hans. Good tips. I'm slowly learning.. :) > 1) that the mongo_conn object itself is thread-safe (does not have internal > state that could get overridden by concurrent calls) and
I'll have to look at the pymongo Connection class and see if it is thread safe in order to verify that mongo_conn is thread-safe. I may post over on the mongodb user group to verify. > 2) that app_globals.mongo_conn was set in a thread-safe way. I assume that > this is happening in some module-level initialization code, which I > *believe* is thread-safe in python (not positive on that point). Yes, mongo_conn is initialized in the lib/app_globals.py Globals __init__, which should be called just on the application load/launch. Ben mentioned above, that is area is thread-safe. > does pylons guarantee a > new instance of your controller object for handling every request (and > hence, per thread)? I think that is a key question ... so I searched around a bit more - I found this (see Mike Orr's post) http://groups.google.com/group/paste-users/browse_thread/thread/bcf2ed96f6581f52 ""... Pylons assumes its native controllers are not thread safe, and instantiates one for each request. ..."" So, from that I assume Pylons instantiates a new controller instance per request. What I've gathered thus far: each controller will get its own instance of Mongo assigned to self.db. However, all self.db Mongo instances share the same reference to the connection object mongo_conn. So, if something is not thread safe with the connection then I may see thread related problems (with saving/updating etc). Does that sound right? I guess, my next step is to determing in pymongo's connection is thread safe or if I need to use thread local. (I've been looking at sqlalchemy/pylons code to figure out how they do threading and whatnot with the model, but for the uninitiated it can be a little confusing if you don't know what you're looking for.) Thanks, Chris On Sep 15, 12:44 pm, Hans Lellelid <[email protected]> wrote: > Hi Chris, > > > > > def __call__(self, environ, start_response): > > """ base.py controller. Get mongodb site context aware object > > based on subdomain """ > > conn = environ['pylons.pylons'].app_globals.mongo_conn > > site = get_site_context(environ) > > self.db = Mongo(mongo_conn, 'myproject', site) # subclass of > > pymongo.database, with site context awareness > > return WSGIController.__call__(self, environ, start_response) > > > Here the Mongo(mongo_conn, 'myproject', site) is just a simple > > subclass of the pymongo.database, but is aware of the requesting site/ > > subdomain. This way, if I say self.db.events.save({name:'Concert'}), > > self.db knows how to save that document to the requesting site's > > collection. (my mongo collections are segregated by site/subdomain). > > > From my questionable testing, it seems to be thread safe, that is, > > docs are getting saved to proper collections. From googling around > > though, it seems I may need to use something like threading.local()? > > Can anybody point me on the right direction on this? Sorry.. not very > > well versed in threading. > > I am by no means an expert in thread-safety, but I'll take a stab at > answering here. I'm also not a Pylons expert, so hopefully if I say > something wrong here someone will correct me :) > > Basically the thing you need to be concerned about with thread safety is > *shared data* -- things like module-level globals are dangerous, for > example, since it's very possible they could be referenced by different > threads concurrently. *local* variables within functions -- e.g. inside > your __call__() method -- do not need to be thread safe, because only one > thread will ever be in that particular execution context at a time. (Sure > multiple threads could be invoking __call__() concurrently, but they'll > each have their own local vars.) > > So, in looking at that code, I would mention a few things. First a couple > assumptions: > > 1) that the mongo_conn object itself is thread-safe (does not have internal > state that could get overridden by concurrent calls) and > 2) that app_globals.mongo_conn was set in a thread-safe way. I assume that > this is happening in some module-level initialization code, which I > *believe* is thread-safe in python (not positive on that point). > > Also, you are setting something into the controller's instance: > > self.db = Mongo(mongo_conn, 'myproject', site) > > Is that sub-class also thread safe? It seems likely that you could have > two different threads in __call__() that are each setting different > yourapp.Mongo instances to the same controller instance var (self.db). > Here's where my knowledge of Pylons is a bit thin: does pylons guarantee a > new instance of your controller object for handling every request (and > hence, per thread)? It may very well, in which case you can ignore my > concerns :) If it doesn't, though, you should probably consider using > thread-local storage for your Mongo instance. You could probably use the > sqlalchemy model for that. > > Hans --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "pylons-discuss" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~----------~----~----~----~------~----~------~--~---
