Re: Shared memory across processes

2007-06-27 Thread Graham Dumpleton
On Jun 27, 8:07 pm, Robert Coup <[EMAIL PROTECTED]> wrote: > Marty Alchin wrote: > > Regardless though, I think Jacob makes the best point so far: Django's > > cache system is robust enough to handle it if you pick a decent > > backend. And if there's a need to make the built-in options more > >

Re: Shared memory across processes

2007-06-27 Thread Nicola Larosa
Marty Alchin wrote: > I remembered seeing that FastCGI can not only run as prefork, Thank goodness. > but defaults to it. Thank *goodness*. One day mutable-shared-state, preemptive multithreading will be looked down on as the ugly, awful, historical accident that it is. The signs are good

Re: Shared memory across processes

2007-06-27 Thread Marty Alchin
> > The biggest hurdle to dbsettings at this point is that it caches > > settings in a standard Python module, which only exists within a > > single process. This was all well and good while developing it, but > > people are now starting to try it under mod_python, which (depending > > on the

Re: Shared memory across processes

2007-06-27 Thread Forest Bond
On Wed, Jun 27, 2007 at 02:41:15PM -, Donny wrote: > > On Jun 26, 11:28 am, "Marty Alchin" <[EMAIL PROTECTED]> wrote: > > The biggest hurdle to dbsettings at this point is that it caches > > settings in a standard Python module, which only exists within a > > single process. This was all well

Re: Shared memory across processes

2007-06-27 Thread Donny
On Jun 26, 11:28 am, "Marty Alchin" <[EMAIL PROTECTED]> wrote: > The biggest hurdle to dbsettings at this point is that it caches > settings in a standard Python module, which only exists within a > single process. This was all well and good while developing it, but > people are now starting to

Re: Shared memory across processes

2007-06-27 Thread Robert Coup
Marty Alchin wrote: > Regardless though, I think Jacob makes the best point so far: Django's > cache system is robust enough to handle it if you pick a decent > backend. And if there's a need to make the built-in options more > robust, we can deal with that when the need arises. > What about

Re: Shared memory across processes

2007-06-26 Thread Marty Alchin
On 6/26/07, James Bennett <[EMAIL PROTECTED]> wrote: > The docs claim the locmem backend is multi-process. Am I misreading that? Hrm, no, you're not misreading it. I hadn't noticed that line before you mentioned it, but I didn't see anything in the code that looked like it handled the

Re: Shared memory across processes

2007-06-26 Thread James Bennett
On 6/26/07, Marty Alchin <[EMAIL PROTECTED]> wrote: > Except that if people aren't using memcached, but locmem, for example, > invalidating the cache only does so for that current process, which > would result in the same problem we're currently having. That is, > unless I'm reading the code

Re: Shared memory across processes

2007-06-26 Thread Graham Dumpleton
On Jun 27, 6:31 am, "Marty Alchin" <[EMAIL PROTECTED]> wrote: > On 6/26/07, Jay Parlar <[EMAIL PROTECTED]> wrote: > > > > That said, mod_python does provide its own autoreloading import > > > system[1], which could probably be used for this, I suppose. By doing > > > it that way, it would only

Re: Shared memory across processes

2007-06-26 Thread Marty Alchin
On 6/26/07, Jacob Kaplan-Moss <[EMAIL PROTECTED]> wrote: > Seems to me that this is exactly the problem that Django's cache > framework was designed to solve. I don't see a reason to reinvent the > wheel for dbsettings... > > It should be extremely simple to invalidate the cache if/when the >

Re: Shared memory across processes

2007-06-26 Thread Marty Alchin
On 6/26/07, Jay Parlar <[EMAIL PROTECTED]> wrote: > > That said, mod_python does provide its own autoreloading import > > system[1], which could probably be used for this, I suppose. By doing > > it that way, it would only reload the one file, which should happen > > pretty quickly. It's

Re: Shared memory across processes

2007-06-26 Thread Waylan Limberg
Another problem with a file is that you generally have to rewrite the entire file to update just one value. Suppose process A has read the filed, then process B reads the file. Then process A updates value FOO and overwrites the file. Process B then updates value BAR and overwrites the file. The

Re: Shared memory across processes

2007-06-26 Thread Jacob Kaplan-Moss
Seems to me that this is exactly the problem that Django's cache framework was designed to solve. I don't see a reason to reinvent the wheel for dbsettings... It should be extremely simple to invalidate the cache if/when the setting is changed, and if people are using memcached like all good

Re: Shared memory across processes

2007-06-26 Thread Jay Parlar
On 6/26/07, Marty Alchin <[EMAIL PROTECTED]> wrote: > I could use a file as cache, yes, but what you're dsecribing wouldn't > really work well in production environments. To my knowledge, Django > reloads itself in its entirety if any loaded modules are changed, not > just a single file. Sure,

Re: Shared memory across processes

2007-06-26 Thread Marty Alchin
On 6/26/07, David Danier <[EMAIL PROTECTED]> wrote: > Or why not use a python-file as "cache"? AFAIK Django already reloads > the settings-file if is has changes. So why not use this and put the > settings generated from the DB there (or in some file imported in > settings.py). A

Re: Shared memory across processes

2007-06-26 Thread David Danier
> I have not yet used your app (although I had intended to until I read > this) so I assumed you had worked this out already. Same goes for me, but I try to answer something useful anyway. > The thing is, if > each process has to look to a central location to retrieve/update, why > not use the

Re: Shared memory across processes

2007-06-26 Thread Marty Alchin
On 6/26/07, Waylan Limberg <[EMAIL PROTECTED]> wrote: > I have not yet used your app (although I had intended to until I read > this) so I assumed you had worked this out already. The thing is, if > each process has to look to a central location to retrieve/update, why > not use the db as that

Re: Shared memory across processes

2007-06-26 Thread Waylan Limberg
Carl, Thats application settings which are stored in a db (thus dbsettings - with no space). See http://code.google.com/p/django-values/ for more info. Marty, I have not yet used your app (although I had intended to until I read this) so I assumed you had worked this out already. The thing is,

Re: Shared memory across processes

2007-06-26 Thread Marty Alchin
On 6/26/07, Carl Karsten <[EMAIL PROTECTED]> wrote: > Forgive my ignorance, but this doesn't seem like a problem that will come up > too > often, so the solution is going to be pretty custom. > > Why would you be changing db settings on the fly? I've put together an app called dbsettings[1]

Re: Shared memory across processes

2007-06-26 Thread Carl Karsten
Marty Alchin wrote: > I expect this isn't the best way to ask this, but this is where the > dbsettings saga has played out so far, and you guys have a good idea > of what I'm trying to do, so I'm asking anyway. > > The biggest hurdle to dbsettings at this point is that it caches > settings in a

Re: Shared memory across processes

2007-06-26 Thread Marty Alchin
On 6/26/07, Jeremy Dunck <[EMAIL PROTECTED]> wrote: > Assuming I've understood the issue, and if you don't expect concurrent > writes, consider a BDB. > > Also, these may or may not be interesting: > http://www.dekorte.com/projects/opensource/ Well, the dekorte stuff looks interesting, but I'm

Re: Shared memory across processes

2007-06-26 Thread Jeremy Dunck
On 6/26/07, Marty Alchin <[EMAIL PROTECTED]> wrote: ... > Has anybody on here ever had a need to do something like this? If so, > are there other decent solutions available? Assuming I've understood the issue, and if you don't expect concurrent writes, consider a BDB. Also, these may or may not

Shared memory across processes

2007-06-26 Thread Marty Alchin
I expect this isn't the best way to ask this, but this is where the dbsettings saga has played out so far, and you guys have a good idea of what I'm trying to do, so I'm asking anyway. The biggest hurdle to dbsettings at this point is that it caches settings in a standard Python module, which