RE: Shared memory caching revisited (was it's supposed to SHARE it, not make more!)
The _session_id is used as the seed for the locking semaphore. *IF* I understood the requirements correctly, the _session_id has to be the same FOR EVERY PROCESS in order for the locking to work as desired, for a given shared data structure. Only if you want to lock the whole thing, rather than a single record. Cache::Cache typically updates just one record at a time, not the whole data structure, so you should only need to lock that one record. Uhh... good point, except that I don't trust the Cache code. The AUTHOR isn't ready to put his stamp of approval on the locking/updating. I'm running 10 hits/sec on this server, and last write wins, which ELIMINATES other writes, is not acceptable. I had a quick look at your code and it seems redundant with Cache::Cache. You're using the locking just to ensure safe updates, which is already done for you. Well, for a single, atomic lock, maybe. My two points above are the why of my hesitancy. Additionally, what if I decide to add to my handler? What if I update more than one thing at once? Now I've got the skeleton based on something that somebody trusts (A:S:L:S), vs what somebody thinks is alpha/beta (C:SASMC). In other words TIMTOWTDI! :-) L8r, Rob
RE: Shared memory caching revisited (was it's supposed to SHARE it, not make more!)
Uhh... good point, except that I don't trust the Cache code. The AUTHOR isn't ready to put his stamp of approval on the locking/updating. That sort of hesitancy is typical of CPAN. I wouldn't worry about it. I think I remember Randal saying he helped a bit with that part. In my opinion, there is no good reason to think that the Apache::Session locking code is in better shape than the Cache::Cache locking, unless you've personally reviewed the code in both modules. Well, the fact is, I respect your opinion. And YES, it seems like I'm doing more work than is probably necessary. I've been screwed over SO MANY TIMES by MYSELF not thinking of some little detail, than I've developed a tendency to design in redundant design redundancy :-) so that if one thing fails, the other will catch it. This reduces downtime... I'm running 10 hits/sec on this server, and last write wins, which ELIMINATES other writes, is not acceptable. As far as I can see, that's all that your code is doing. You're simply locking when you write, in order to prevent corruption. You aren't acquiring an exclusive lock when you read, so anyone could come in between your read and write and make an update which would get overwritten when you write, i.e. last write wins. Again, good point... I'm coding as if the WHOLE cache structure will break if any little thing gets out of line. I was trying to think in terms of data safety like one would with threading, because A) I was worried about weather shared memory was as sensitive to locks/corruption as threading, and B) I reviewed Apache::Session's lock code, but didn't review Cache::Cache's (20/20 hindsight, ya know). You're more than welcome to roll your own solution based on your personal preferences, but I don't want people to get the wrong idea about Cache::Cache. It handles the basic locking needed for safe updates. Then my code just got waaay simpler, both in terms of data flow and individual coding sections. THANK YOU! :-) L8r, Rob #!/usr/bin/perl -w use Disclaimer qw/:standard/;
Re: Shared memory caching revisited (was it's supposed to SHARE it, not make more!)
On Tue, Sep 04, 2001 at 12:14:52PM -0700, Rob Bloodgood wrote: ***OH WOW!*** So, DURING the course of composing this message, I've realized that the function expire_old_accounts() is now redundant! Cache::Cache takes care of that, both with expires_in and max_size. I'm leaving it in for reference, just to show how it's improved. :-) [snip] use Apache::Session::Lock::Semaphore (); use Cache::SizeAwareSharedMemoryCache (); # this is used in %cache_options, as well as for locking use constant SIGNATURE = 'EXIT'; use constant MAX_ACCOUNTS = 300; # use vars qw/%ACCOUNTS/; use vars qw/$ACCOUNTS $locker/; my %cache_options = ( namespace = SIGNATURE, default_expires_in = max_size = MAX_ACCOUNTS ); Very neat thought about how to use max_size to limit the the accounts! Unfortunately, you demonstrated that I did a *terrible* job at documenting what size means. It means size in bytes, not items. I will add max_items and limit_items to the TODO list. In the meantime, I will improve the documentation. -DeWitt
RE: Shared memory caching revisited (was it's supposed to SHARE it, not make more!)
What about my IPC::FsSharevars? I've once mentioned it on this list, but I don't have the time to read all list mail, so maybe I've missed some conclusions following the discussion from last time. I remember the post and went to find IPC::FsSharevars a while ago and was un-intrigued when I didn't find it on CPAN. has there been any feedback from the normal perl module forums? --Geoff
RE: Shared memory caching revisited (was it's supposed to SHARE it, not make more!)
At 20:37 Uhr -0400 4.9.2001, Geoffrey Young wrote: I remember the post and went to find IPC::FsSharevars a while ago and was un-intrigued when I didn't find it on CPAN. has there been any feedback from the normal perl module forums? I haven't announced it on other forums (yet). (I think it's more of a working version yet that needs feedback and some work to make it generally useable (i.e. under mod_perl). Which forum should I post on?) christian
Re: Shared memory caching revisited (was it's supposed to SHARE it, not make more!)
Christian == Christian Jaeger [EMAIL PROTECTED] writes: Christian I haven't announced it on other forums (yet). (I think it's Christian more of a working version yet that needs feedback and some Christian work to make it generally useable (i.e. under Christian mod_perl). Which forum should I post on?) If you put it on the CPAN with a version number below 1, that's usually a clue that it's still alpha or beta. Then you can announce it through the normal module announcement structures. If you hide it, I'm sure not installing it. -- Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095 [EMAIL PROTECTED] URL:http://www.stonehenge.com/merlyn/ Perl/Unix/security consulting, Technical writing, Comedy, etc. etc. See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!
Re: Shared memory caching revisited (was it's supposed to SHARE it, not make more!)
Perrin == Perrin Harkins [EMAIL PROTECTED] writes: Uhh... good point, except that I don't trust the Cache code. The AUTHOR isn't ready to put his stamp of approval on the locking/updating. Perrin That sort of hesitancy is typical of CPAN. I wouldn't worry Perrin about it. I think I remember Randal saying he helped a bit Perrin with that part. I helped with the code that ensures that *file* writes are atomic updates. I taught DeWitt the trick of writing to a temp file, then renaming when ready, so that any readers see only the old file or the new file, but never a partially written file. I don't think Cache::Cache has enough logic for an atomic read-modify-write in any of its modes to implement (for example) a web hit counter. It has only atomic write. The last write wins strategy is fine for caching, but not for transacting, so I can see why Rob is a bit puzzled. It'd be nice if we could build a generic atomic read-modify-write, but now we're back to Apache::Session, which in spite of its name works fine away from Apache. :) Caching. An area of interest of mine, but I still don't seem to get around to really writing the framework I want, so all I can do is keep lobbing grenades into the parts I don't want. :) :) Sorry guys. :) -- Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095 [EMAIL PROTECTED] URL:http://www.stonehenge.com/merlyn/ Perl/Unix/security consulting, Technical writing, Comedy, etc. etc. See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!
Re: Shared memory caching revisited (was it's supposed to SHARE it, not make more!)
I don't think Cache::Cache has enough logic for an atomic read-modify-write in any of its modes to implement (for example) a web hit counter. It has only atomic write. The last write wins strategy is fine for caching, but not for transacting, so I can see why Rob is a bit puzzled. In his example code he was only doing atomic writes as well, so it should work at least as well for his app as what he had before. It'd be nice if we could build a generic atomic read-modify-write Maybe a get_for_update() method is what's needed. It would block any other process from doing a set() or a get_for_update() until the set() for that key has completed. It's still just advisory locking though, so if you forget and use a regular get() for some data you later plan to set(), you will not be getting atomic read-modify-write. Maybe get() could be re-named to get_read_only(), or set a flag that prevents saving the fetched data. Most caching apps are happy with last save wins though, so I guess anything like that would need to be optional. - Perrin
