Re: Caching and model import question
Well, I was wrong... how *should* this work? I tried a signal handler to reload modules whenever someone added or deleted a model, but that only seems to matter if a particular module that loads "models" hasn't been loaded yet. On Wed, Mar 21, 2012 at 12:59 PM, Jeff Heard wrote: > Thanks. I don't think that *in principle* creating new models on the > fly is a bad thing, although maybe some will argue with me. There's > another Django app that allows you to do this within the admin > interface, and I've considered using it. I dislike the idea of having > to make assumptions about the backend that will be running, though, > and assuming that my users will be using mod_wsgi, FastCGI, or > gunicorn is anaethema to the idea of making a portable app. I *think* > I have it figured, out, although my implementation still feels > "hackish" to me. I suppose another couple of iterations on it may > make sense. > > On Wed, Mar 21, 2012 at 12:52 PM, Tom Evans wrote: >> On Wed, Mar 21, 2012 at 4:19 PM, Jeff Heard >> wrote: >>> Question 1: Would this work with views, or for that matter, anything >>> else in Django, assuming you're using a WSGI server like gunicorn? >>> http://code.activestate.com/recipes/578078/ Specifically, will it >>> cache across HTTP requests, or not? >>> >> >> Probably, but that is the worst place to cache in a web framework, as >> it is a cache per process. That is 'ok' if you use a single process, >> multi thread serving model on a single server, but useless/wasteful if >> you run a multi-process serving model or serve from multiple servers. >> >> It is much better to use caches at a higher level, eg memcached, which >> is distributed and can be shared easily amongst any model. >> >>> I guess the one thing I don't understand well in Django is when >>> modules are re-loaded. I know it's different in a "full-fledged" >>> setup vs. the test server, but is there a rule one can follow? Is it >>> different on WSGI vs. FastCGI? >>> >> >> Never automatically. mod_wsgi will reload your app if you touch >> (change the mtime) the app.wsgi file. mod_fastcgi will reload your app >> when you restart the process. >> >>> And finally, if I delete a module from sys.modules, does that >>> effectively delete it from the cache or is there something else I need >>> to do to make sure it's reloaded the next time someone makes a >>> request? I would assume that deleting the module from sys.modules >>> would only delete it from one worker process. Is there an accepted >>> way to get a module to reload across all processes? >>> >>> What I'm trying to do is allow a user to create models on the fly by >>> uploading data. One answer is just to have a post_save signal call >>> supervisorctl and restart the webservices, but that seems nonportable >>> and rather broken. There's got to be a better way. >>> >> >> Eurgh. Now I don't want to help you! These links will help: >> >> http://code.google.com/p/modwsgi/wiki/ReloadingSourceCode#Reloading_In_Daemon_Mode >> http://code.google.com/p/modwsgi/wiki/ReloadingSourceCode#Restarting_Daemon_Processes >> >> Cheers >> >> Tom >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Django users" group. >> To post to this group, send email to django-users@googlegroups.com. >> To unsubscribe from this group, send email to >> django-users+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/django-users?hl=en. >> -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: Caching and model import question
Thanks. I don't think that *in principle* creating new models on the fly is a bad thing, although maybe some will argue with me. There's another Django app that allows you to do this within the admin interface, and I've considered using it. I dislike the idea of having to make assumptions about the backend that will be running, though, and assuming that my users will be using mod_wsgi, FastCGI, or gunicorn is anaethema to the idea of making a portable app. I *think* I have it figured, out, although my implementation still feels "hackish" to me. I suppose another couple of iterations on it may make sense. On Wed, Mar 21, 2012 at 12:52 PM, Tom Evans wrote: > On Wed, Mar 21, 2012 at 4:19 PM, Jeff Heard > wrote: >> Question 1: Would this work with views, or for that matter, anything >> else in Django, assuming you're using a WSGI server like gunicorn? >> http://code.activestate.com/recipes/578078/ Specifically, will it >> cache across HTTP requests, or not? >> > > Probably, but that is the worst place to cache in a web framework, as > it is a cache per process. That is 'ok' if you use a single process, > multi thread serving model on a single server, but useless/wasteful if > you run a multi-process serving model or serve from multiple servers. > > It is much better to use caches at a higher level, eg memcached, which > is distributed and can be shared easily amongst any model. > >> I guess the one thing I don't understand well in Django is when >> modules are re-loaded. I know it's different in a "full-fledged" >> setup vs. the test server, but is there a rule one can follow? Is it >> different on WSGI vs. FastCGI? >> > > Never automatically. mod_wsgi will reload your app if you touch > (change the mtime) the app.wsgi file. mod_fastcgi will reload your app > when you restart the process. > >> And finally, if I delete a module from sys.modules, does that >> effectively delete it from the cache or is there something else I need >> to do to make sure it's reloaded the next time someone makes a >> request? I would assume that deleting the module from sys.modules >> would only delete it from one worker process. Is there an accepted >> way to get a module to reload across all processes? >> >> What I'm trying to do is allow a user to create models on the fly by >> uploading data. One answer is just to have a post_save signal call >> supervisorctl and restart the webservices, but that seems nonportable >> and rather broken. There's got to be a better way. >> > > Eurgh. Now I don't want to help you! These links will help: > > http://code.google.com/p/modwsgi/wiki/ReloadingSourceCode#Reloading_In_Daemon_Mode > http://code.google.com/p/modwsgi/wiki/ReloadingSourceCode#Restarting_Daemon_Processes > > Cheers > > Tom > > -- > You received this message because you are subscribed to the Google Groups > "Django users" group. > To post to this group, send email to django-users@googlegroups.com. > To unsubscribe from this group, send email to > django-users+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/django-users?hl=en. > -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: Caching and model import question
On Wed, Mar 21, 2012 at 4:19 PM, Jeff Heard wrote: > Question 1: Would this work with views, or for that matter, anything > else in Django, assuming you're using a WSGI server like gunicorn? > http://code.activestate.com/recipes/578078/ Specifically, will it > cache across HTTP requests, or not? > Probably, but that is the worst place to cache in a web framework, as it is a cache per process. That is 'ok' if you use a single process, multi thread serving model on a single server, but useless/wasteful if you run a multi-process serving model or serve from multiple servers. It is much better to use caches at a higher level, eg memcached, which is distributed and can be shared easily amongst any model. > I guess the one thing I don't understand well in Django is when > modules are re-loaded. I know it's different in a "full-fledged" > setup vs. the test server, but is there a rule one can follow? Is it > different on WSGI vs. FastCGI? > Never automatically. mod_wsgi will reload your app if you touch (change the mtime) the app.wsgi file. mod_fastcgi will reload your app when you restart the process. > And finally, if I delete a module from sys.modules, does that > effectively delete it from the cache or is there something else I need > to do to make sure it's reloaded the next time someone makes a > request? I would assume that deleting the module from sys.modules > would only delete it from one worker process. Is there an accepted > way to get a module to reload across all processes? > > What I'm trying to do is allow a user to create models on the fly by > uploading data. One answer is just to have a post_save signal call > supervisorctl and restart the webservices, but that seems nonportable > and rather broken. There's got to be a better way. > Eurgh. Now I don't want to help you! These links will help: http://code.google.com/p/modwsgi/wiki/ReloadingSourceCode#Reloading_In_Daemon_Mode http://code.google.com/p/modwsgi/wiki/ReloadingSourceCode#Restarting_Daemon_Processes Cheers Tom -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Caching and model import question
Question 1: Would this work with views, or for that matter, anything else in Django, assuming you're using a WSGI server like gunicorn? http://code.activestate.com/recipes/578078/ Specifically, will it cache across HTTP requests, or not? I guess the one thing I don't understand well in Django is when modules are re-loaded. I know it's different in a "full-fledged" setup vs. the test server, but is there a rule one can follow? Is it different on WSGI vs. FastCGI? And finally, if I delete a module from sys.modules, does that effectively delete it from the cache or is there something else I need to do to make sure it's reloaded the next time someone makes a request? I would assume that deleting the module from sys.modules would only delete it from one worker process. Is there an accepted way to get a module to reload across all processes? What I'm trying to do is allow a user to create models on the fly by uploading data. One answer is just to have a post_save signal call supervisorctl and restart the webservices, but that seems nonportable and rather broken. There's got to be a better way. -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
