On Nov 2, 2:58 pm, Graham Dumpleton <[EMAIL PROTECTED]>
wrote:
> On Nov 3, 2:03 am, belred <[EMAIL PROTECTED]> wrote:
>
>
>
> > On Nov 2, 3:29 am, Steve Holden <[EMAIL PROTECTED]> wrote:
>
> > > belred wrote:
> > > > i have an wget event in a cron job. the view does some processing
> > > > which involves calling external websites and databases (takes about 25
> > > > seconds) and updates some module level dictionary lookup variables
> > > > (about 7 MB of data) which the rest of the program reads. but
> > > > unfortunately, only the one apache child process seems to be updated,
> > > > not all the others. how can i share this module level variable across
> > > > processes? this lookup happens multiple times per request. but the
> > > > internal data gets updated nightly.
>
> > > Aah, so this is the real question! I don't believe Django signals are
> > > intended to serve needs like this. How does "the rest of the program"
> > > read the module-level dictionary items? The more explicit you are about
> > > the exact communication mechanism involved the more likely it is someone
> > > (probably not me) will be able to help.
>
> > > regards
> > > Steve
>
> > thanks steve,
>
> > you comments about signals were very clear and should be in the
> > documentation.
>
> > when i receive the nightly wget request to update the data, it's
> > stored in module level dicts which other parts of the program use by
> > simply importing the data module.
>
> > import data
>
> > def view_func(request):
> > foo = data.lookup['foo']
> > ...
>
> > def view_update_data(request):
> > data.lookup = retrieve_new_data()
> > ...
>
> > this works correctly for the child process that receives the update,
> > but the other child process don't seem to be getting the updates.
>
> > i thought after i did an update i could os.uptime() thewsgifile
> > which would cause all of my django project to restart, but it doesn't
> > seem to be updating my data either.
>
> If the module level data you are talking about are just transient
> global variables in the module, then other processes can't see them
> (or other sub interpreters in same process). For a start read:
>
> http://code.google.com/p/modwsgi/wiki/ProcessesAndThreading
>
> Written for mod_wsgi, but equally applicable to mod_python for
> embedded mode description at least. Take note of very last section
> where it summarises data sharing issues.
>
> If it is module level data, only solution would be to use mod_wsgi
> daemon mode and delegate Django instance to run in single process.
>
> In respect of triggering a restart, if the data is indeed transient,
> ie., in memory only, how does it get read on startup if you are
> relying on cron job to trigger its initialisation?
>
> Graham
in the apps __init__.py it sees if a pickle file exists. if so, it
read it in and sets the module level variable. if it doesn't exist,
it call on web services to get new data and process it, setting the
module level variable and saving a pickle. saving the pickle allows
the external server to be down as well as my server to restart and
still use the "last known good data". it also allows my server to
start instantly instead of 25 second delay while retrieving new data.
nightly, cron uses wget to call this routine.
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-users?hl=en
-~----------~----~----~----~------~----~------~--~---
