Re: [ANNOUNCE] Cache::Cache 1.0
On Sat, Apr 13, 2002 at 04:33:11PM +0200, Eric Cholet wrote: I'm seeing this problem when using Apache::SOAP, using Cache::Cache 0.99 or 1.0. Downgrading to Digest::MD5 2.12 indeed solves it. Hi -- I recently released version 1.01 of Cache::Cache to CPAN. This version uses Digest::SHA1 as a substitute for MD5 until the bugs in that library are fixed. If anyone has any issues with 1.0 and MD5, please download 1.01 and let me know if anything goes wrong. Cheers, -DeWitt
[ANNOUNCE] Cache::Cache 1.0
Hi, The following is an announcement of Cache::Cache 1.0. Please read the summary below for more details. I want to ask a special favor of the modperl community (who tend to get a lot of mileage out of this code) -- I'm concerned with the use of Digest::MD5, particularly version 2.16, which is used by the code to generate unique filenames in the file-based cache. I've been seeing some pretty ugly bugs with the the MD5 code, and this has been confirmed on a few mailing lists and CPAN bug reports. Please let me know if you notice any weird behavior with your system after upgrading to version 1.0. (All unit test pass consistently, but under certain conditions, such as when XML::DOM is loaded, I think that Digest::MD5 may always return the same hash, no matter what the key.) Note that you can simply run Digest::MD5 2.12 and not 2.16 if you have any issues in production. So don't worry about upgrading to version 1.0 of Cache::Cache. I'm thinking of switching to Digest::SHA1 in version 1.01 to avoid this until a new version of Digest::MD5 is released. Does anyone know of a reason to avoid that? (SHA1 is slightly slower, but very little time in spent generating keys anyway, so it may not be a big deal.) For more information on the MD5 issue, please see: http://rt.cpan.org/NoAuth/Bug.html?id=362. Thanks to everyone for your help. Cheers, -DeWitt Summary: The Cache modules are designed to assist a developer in persisting data for a specified period of time. Often these modules are used in web applications to store data locally to save repeated and redundant expensive calls to remote machines or databases. People have also been known to use Cache::Cache for its straightforward interface in sharing data between runs of an application or invocations of a CGI-style script or simply as an easy to use abstraction of the filesystem or shared memory. Release Notes: Release of Cache::Cache version 1.0. This version fixes outstanding issues on Win32 platforms and other minor bug fixes. Project Homepage: http://perl-cache.sourceforge.net/ Tar/GZ: http://prdownloads.sourceforge.net/perl-cache/Cache-Cache-1.0.tar.gz http://www.cpan.org/authors/id/D/DC/DCLINTON/Cache-Cache-1.0.tar.gz Changelog: http://sourceforge.net/project/shownotes.php?release_id=83292 CVS tree (viewcvs): http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/perl-cache/ The following is the Cache-Cache-1.0 README file: Copyright (C) 2001 DeWitt Clinton All Rights Reserved You may distribute under the terms of either the GNU General Public License or the Artistic License, as specified in the Perl README file. NAME Cache::Cache DESCRIPTION The Cache modules are designed to assist a developer in persisting data for a specified period of time. Often these modules are used in web applications to store data locally to save repeated and redundant expensive calls to remote machines or databases. People have also been known to use Cache::Cache for its straightforward interface in sharing data between runs of an application or invocations of a CGI-style script or simply as an easy to use abstraction of the filesystem or shared memory. The Cache package provides Cache::Cache, a generic interface for creating persistent data stores. This interface is implemented by the Cache::MemoryCache, Cache::SharedMemoryCache, Cache::FileCache, Cache::SizeAwareFileCache, Cache::SizeAwareMemoryCache, and Cache::SizeAwareSharedMemoryCache classes. This work aggregates and extends the obsolete File::Cache and IPC::Cache projects. REQUIREMENTS Digest::MD5 Error File::Spec File::Path IPC::ShareLite Storable INSTALLATION perl Makefile.PL make make test make install USAGE First, choose the best type of cache implementation for your needs. The simplest cache is the MemoryCache, which is suitable for applications that are serving multiple sequential requests, and wish to avoid making redundant expensive queries, such as an Apache/mod_perl application talking to a database. If you wish to share that data between processes, then perhaps the SharedMemoryCache is appropriate, although its behavior is tightly bound to the underlying IPC mechanism, which varies from system to system, and is unsuitable for large objects or large numbers of objects. When the SharedMemoryCache is not acceptable, then FileCache offers all of the same functionality with similar performance metrics, and it is not limited in terms of the number of objects or their size. If you wish to maintain a strict limit on the size of a file system based cache, then the SizeAwareFileCache is the way to go. Similarly, the SizeAwareMemoryCache and the SizeAwareSharedMemoryCache add size management functionality to the MemoryCache and SharedMemoryCache classes respectively. Using a cache is simple. Here is some sample code for instantiating and using a file system
Re: [ANNOUNCE] Cache::Cache 1.0
On Sun, Apr 07, 2002 at 08:52:59PM +0300, Issac Goldstand wrote: So how exactly is the auto_purge meant to be used - the documentation is very hazy on that point (sure I can patch it - but you have to explain it to me first :)), and I just got lost when looknig at the module source. The auto_purge functionality is designed to enable a cache to delete expired cache entries periodically. While user code could call the purge( ) method every time they access the cache, it would be rather inefficient and often completely unnecessary. Thus the auto_purge code automatically calls purge( ) after a user specified interval of time has elapsed. You set the amount of time between auto_purges by setting the 'auto_purge_interval' option when instantiating the cache. After setting the 'auto_purge_interval' the purge method will be called when a cache instance is created (via 'new') and the time interval has elapsed. Also, by setting 'auto_purge_on_set' to true, the cache will automatically call purge when the 'set' method is called and the time interval has elapsed. Respectively, by setting 'auto_purge_on_get' to true, the cache will automatically call purge when the 'get' method is called and the time interval has elapsed. If you tend to instantiate caches relatively frequently (such as during the initialization of a modperl process), you can probably get away with simply setting the 'auto_purge_interval'. However, if you processes run for a long time and you'd like to automatically call purge( ) more frequently, you can could consider using the 'auto_purge_on_set' option. I'd be surprised if people were using the 'auto_purge_on_get' functionality, as it would most likely be overkill. Also, keep in mind that if you hit the same keys over and over again, then it may be sufficient to rely on 'get' to delete expired entries when they are requested. Hope this helps clarify things! Cheers, -DeWitt
Re: transient object data
On Sat, Dec 22, 2001 at 06:11:33AM -0800, brian moseley wrote: doesn't it seem like there should be a way to denote object data as transient so that it doesn't get serialized by Storable, etc? I'd love that as well. For example, when persisting Cache::Object instances I manually strip out transient (in this case, it is re-creatable) data such as the name of the key and the size of the object, in order to save space in storage. Sounds like Perl 6 needs a generic notion of Serializable. -DeWitt
Re: [ANNOUNCE] Apache::Singleton 0.03
On Sat, Dec 22, 2001 at 10:39:03PM +0900, Tatsuhiko Miyagawa wrote: One instance for one server (across all httpd processes). Implemented using Cache::SharedMemoryCache (IPC). Maybe you want to consider directly using Cache::SharedMemoryBackend instead of the SharedMemoryCache class. The full cache class has the overhead of dealing with object expiration times, which isn't going to be necessary for the Singleton. Also, try experimenting with the FileBackend as your sharing mechanism instead of shared memory. If you have a large number of Singleton objects (unlikely, but possible) then this may be a huge performance win. Cheers, -DeWitt
Re: Comparison of different caching schemes
On Wed, Dec 12, 2001 at 03:05:33PM +1100, Rob Mueller (fastmail) wrote: I sat down the other day and wrote a test script to try out various caching implementations. The script is pretty basic at the moment, I just wanted to get an idea of the performance of different methods. Rob, wow! This is fantastic work! I'm wondering if you could include a test on the Cache::FileBackend class. The Cache::FileCache class is not optimized for the particular type of gets and sets that you were testing for. In fact, each of those requests will still need to go through the process of checking for object expiration, which may add the overhead that you observe in the benchmark. Not that I'm expecting Cache::FileBackend to do significantly better -- it still does name hashing, directory hashing, taint checking, cloning of objects, etc. But I am curious to see what difference it makes. You could try Cache::SharedMemoryBackend as well. In general the Cache::* modules were designed for clarity and ease of use in mind. For example, the modules tend to require absolutely no set-up work on the end user's part and try to be as fail-safe as possible. Thus there is run-time overhead involved. That said, I'm certainly not against performance. :) These benchmarks are going to be tremendously useful in identifying bottlenecks. However, I won't be able to optimize for these particular benchmarks, as Cache::Cache is designed to do something different than straight gets and sets. Again, thank you, Rob. This is great, -DeWitt
Re: [RFC] Apache::CacheContent - Caching PerlFixupHandler
On Tue, Dec 11, 2001 at 02:31:36AM -0800, Paul Lindner wrote: Right. A more elaborate Apache::CacheContent would have a filename hash function, and a separate cache directory structure along the lines of Cache::FileCache. Just curious -- any reason not to use Cache::Cache as the persistance mechanism? It was designed for exactly this scenario and could provide a nice abstraction for the filesystem or shared memory, as well as handle things like filename hashing and branching directories (and namespaces, size limits, OS independance, taint checking, and more). -DeWitt
[ANNOUNCE] Cache::Cache 0.99
Summary:
The Cache modules are designed to assist a developer in persisting
data for a specified period of time. Often these modules are used
in web applications to store data locally to save repeated and
redundant expensive calls to remote machines or databases. People
have also been known to use Cache::Cache for its straightforward
interface in sharing data between runs of an application or
invocations of a CGI-style script or simply as an easy to use
abstraction of the filesystem or shared memory.
Release Notes:
This release contains newly refactored code, updated documentation,
and numerous clarifications and minor bug fixes.
Project Homepage:
http://sourceforge.net/projects/perl-cache/
Tar/GZ:
http://prdownloads.sourceforge.net/perl-cache/Cache-Cache-0.99.tar.gz
Changelog:
http://sourceforge.net/project/shownotes.php?release_id=64913
CVS tree (viewcvs):
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/perl-cache/
The following is the Cache-Cache-0.99 README file:
Copyright (C) 2001 DeWitt Clinton All Rights Reserved
You may distribute under the terms of either the GNU General Public
License or the Artistic License, as specified in the Perl README file.
NAME
Cache::Cache
DESCRIPTION
The Cache modules are designed to assist a developer in persisting
data for a specified period of time. Often these modules are used
in web applications to store data locally to save repeated and
redundant expensive calls to remote machines or databases. People
have also been known to use Cache::Cache for its straightforward
interface in sharing data between runs of an application or
invocations of a CGI-style script or simply as an easy to use
abstraction of the filesystem or shared memory.
The Cache package provides Cache::Cache, a generic interface
for creating persistent data stores. This interface is implemented
by the Cache::MemoryCache, Cache::SharedMemoryCache, Cache::FileCache,
Cache::SizeAwareFileCache, Cache::SizeAwareMemoryCache, and
Cache::SizeAwareSharedMemoryCache classes.
This work aggregates and extends the obsolete File::Cache and
IPC::Cache projects.
REQUIREMENTS
Digest::MD5
Error
File::Spec
File::Path
IPC::ShareLite
Storable
INSTALLATION
perl Makefile.PL
make
make test
make install
USAGE
First, choose the best type of cache implementation for your needs.
The simplest cache is the MemoryCache, which is suitable for
applications that are serving multiple sequential requests, and
which to avoid making redundant expensive queries, such as an
Apache/mod_perl application talking to a database. If you wish to
share that data between processes, then perhaps the
SharedMemoryCache is appropriate, although its behavior is tightly
bound to the underlying IPC mechanism, which varies from system to
system, and is unsuitable for large objects or large numbers of
objects. When the SharedMemoryCache is not acceptable, then
FileCache offers all of the same functionality with similar
performance metrics, and it is not limited in terms of the number of
objects or their size. If you wish to maintain a strict limit on
the size of a file system based cache, then the SizeAwareFileCache
is the way to go. Similarly, the SizeAwareMemoryCache and the
SizeAwareSharedMemoryCache add size management functionality
to the MemoryCache and SharedMemoryCache classes respectively.
Using a cache is simple. Here is some sample code for instantiating
and using a file system based cache.
use Cache::FileCache;
my $cache = new Cache::FileCache( );
my $customer = $cache-get( $name );
if ( not defined $customer )
{
$customer = get_customer_from_db( $name );
$cache-set( $name, $customer, 10 minutes );
}
return $customer;
Please refer to the perldoc for Cache::Cache and the related
implementations for complete documentation.
INCOMPATIBLE CHANGES
Cache::Cache 0.99 contains the following incompatible changes:
* Error::Simple is thrown on exceptions
* the get_identifiers method has been deprecated in favor of the
get_keys method
* the internal format of object in a FileCache has been modified,
necessitating a clearing of the cache while upgrading (make test
does this for the standard cache root)
SEE ALSO
The project homepage at http://perl-cache.sourceforge.net
AUTHOR
Original author: DeWitt Clinton [EMAIL PROTECTED]
Copyright (C) 2001 DeWitt Clinton
Re: Doing Authorization using mod_perl from a programmers perspective
On Mon, Nov 19, 2001 at 07:51:55AM -0800, Randal L. Schwartz wrote: But this is obvious. I'm confused about why I'd have to explain it. :( I posted this a year or two back: [EMAIL PROTECTED]">http://mathforum.org/epigone/modperl/jytwortwor/[EMAIL PROTECTED] Here is the relevant part of that post: It makes sense to start with the requirements for what it means to implement those secure features. My requirements have an obvious e-commerce bias, and should probably be heavily reviewed by anyone thinking of using this design for online banking or government work. First, I'd like to introduce the notion of a secure session. Secure sessions have some subtle differences from the traditional notion of the session, and I'll point those out when they appear. The secure session has the following properties: *) The user is able to initiate a secure session by providing proper credentials (i.e., a username and password pair) via a login process. *) The user is able to terminate the secure session via a logout process. *) Secure sessions must be able to time out automatically. *) Secure sessions must *never* transmit sensitive data (such as the password) over insecure channels. *) The secure session, while it requires the use of a secure protocol (such as HTTPS), should not require the use of cookies. Cookies, however, can be employed to extend the functionality of the system. Additionally, I feel that one of the essential requirements to any enterprise quality, highly scalable, and fault-tolerant system is the ability to store session state on a tier other than the front-end. This usually means storing state in a shared database, but there are other options. A very effective method of meeting the requirements above is to use a two token architecture. The first token is a user identifier, which can be passed over insecure channels. This user identification token can be used to restore state that isn't particularly sensitive, such as a shopping cart or user preferences. The second token is a secure identifier, which is never passed over insecure channels. The secure identifier is required to access sensitive data, such as credit card data. (It is possible to create an architecture that uses *only* the secure token, but there are significant benefits in terms of flexibility afforded by using two tokens, so I won't go into the one token model here.) The fundamental goal of the secure token is to a) keep it safe from prying eyes, and b) use it is a requirement for accessing secure data. Tokens can be passed in one of two ways. First, the token can be passed as part of the URL. Second, the token can be passed in a cookie. I've found it very useful to create an initialization procedure inside applications that check both places for these tokens, and abstract the particular mechanism used to pass them. Note, however, that is important to remember how the token was passed -- if it came from a cookie, it is cosmetic to not pass it around in the URL. I also recommend creating a library used in your templates that has a function to build URLs that automatically append the tokens if necessary. Since it is critical that secure tokens are never passed in the clear, it really helps to have that function, because it can prevent the secure token from being sent via an insecure page in a URL (i.e., only setting secure tokens in URLs that contain https). The user identification token is typically used to restore state from the database. It is created whenever a page is viewed and the client has either not provided one (via the URL or a cookie), or the token they provide doesn't exist in the database. This token should then be set in both the URL and in a cookie. On the subsequent page request, if the user identification token is found in a cookie, it is probably safe to stop putting it in the URL. However, if the token is found only in the URL (probably meaning the client does not use cookies) the server should not try writing the cookie again. The advantage of the cookie is such that the user can insecurely identify themselves across browser sessions. This is perfectly acceptable when used to restore some insensitive state data (such as a name or shopping cart). However, the system still functions without the cookie. The user can now login to associate their client with particular stored user information. The login process is as follows: 1) The client connects to a secure page that presents a form requesting a username and password. (There has been much discussion about whether a HTTP form that POSTS to a secure page will encrypt the posted data. There is too much ambiguity here -- I recommend using HTTPS on the login form as well.) The ACTION of the form must also be a secure page. 2) On receiving the username and password, the server compares an encrypted version of the password with the stored encrypted password associated with that customer. If the passwords do not match,
Re: Cache::* and MD5 collisions [was: [OT] Data store options]
On Thu, Nov 08, 2001 at 01:11:21PM -0500, Barrie Slaymaker wrote: Even a bit more OT: one thing to watch out for, especially if you plan on caching a *lot* of data, is that the Cache::* modules did not do collision detection on MD5 collisions the last time I looked. Forgive me if that's changed recently. I knew that a collision was technically possible, but the odds seemed astronomically low. But if security is a concern, then I agree that MD5-ing the key isn't the best way to create a unique identifier. The thing that I liked most about MD5 (other than its convenience) is that it tends to distribute very evenly. For example, file system caches fill their directories roughly equally when their paths are created from MD5 hashed keys. Doing something simple and unique like URL-encoding the key to make a legal identifier (legal in the sense that it is a valid filename) wouldn't distribute as evenly. Barrie raised this question privately to me earlier this year, and this is what I wrote at the time: Good question. They would overwrite each other. You may want to check out this RFC for more information about the odds of that: http://www.faqs.org/rfcs/rfc1321.html Specifically, they say that it is conjectured that the difficulty of coming up with two messages having the same message digest is on the order of 2^64 operations, and that the difficulty of coming up with any message having a given message digest is on the order of 2^128 operations. This means you would need 18,446,744,073,709,551,616 keys to make it 100% likely that there would be a collision. Assuming you were only willing to risk a 0.01% chance that there is a collision, you could still fit 1,844,674,407,370,955 keys. Since each value stored will take at least 1k in memory, that number of keys would require 1,844,674 Terrabytes of storage. In other words, it is highly unlikely that you could get a collision in real world usage. Note that I'm not disagreeing with Barrie at all -- I just took the easy way out and refuted the likelihood of an infinite number of monkeys hitting my code. :) -DeWitt
Re: Using DOM to build your output documents
On Wed, Oct 03, 2001 at 04:43:41PM -0700, Jeffrey W. Baker wrote: I'd also like to hear people's opinions on XML::DOM (I think it stinks and I've replaced it with a C DOM implementation). You just proved that you already know everything you need to on the subject of XML::DOM. :) Seriously though, what C DOM implementation did you choose and how did you integrate it with your Perl code? I've long since abandoned XML::DOM and use things XML::Parser::PerlSAX instead, but that doesn't achieve what you are talking about vis-a-vis building up documents element by element. For output generation, I still tend to push data into a template engine such as Embperl or Template::Toolkit and use embedded control structures to manipulate that data. Interestingly, the company I used to work at (eZiba) just ported their front-end over to Mason and ended up somewhere in-between -- i.e., reusable page components that they build up at run-time. Check out the Avacet's (www.avacet.com) TemplateService for an example of the push model in action. Cheers, -DeWitt
Re: MSIISProbes.pm v1.03
On Fri, Sep 28, 2001 at 08:49:22AM -0700, Nick Tonkin wrote: Cache::FileCache defaults to using /tmp for the location of its cache; does the system have /tmp (not sure what Cache::FileCache does if there's no /tmp, hafta look at the code). You can manually override the temp directory by setting the 'cache_root' option when instantiating the cache. If cache_root isn't set, then File::Spec's tmpdir( ) routine will be called, which seems to return a value on just about all the machines I've tested (judging by the lack of temp directory bug reports). Cheers, -DeWitt
[ANNOUNCE] Cache::Cache 0.09
[Note: A big thanks to everyone on modperl that has provided
invaluable insight and support for this release.]
Summary:
The Perl Cache package provides Cache::Cache, a generic interface
for creating persistent data stores. This interface is implemented
by the Cache::MemoryCache, Cache::SharedMemoryCache, Cache::FileCache,
Cache::SizeAwareFileCache, Cache::SizeAwareMemoryCache, and
Cache::SizeAwareSharedMemoryCache classes. This work replaces
File::Cache and IPC::Cache.
Release Notes:
This release fixes outstanding bugs in cache instantiation and race
condition handling, and is the first release under the dual
GPL/Artistic license.
Project Homepage:
http://sourceforge.net/projects/perl-cache/
Tar/GZ:
http://www.cpan.org/modules/by-module/Cache/Cache-Cache-0.09.tar.gz
Changelog:
http://sourceforge.net/project/shownotes.php?release_id=52255
- applied Axel Beckert patch to fix the expiration units
- applied Ken Williams's directory creation patch to pass all tests
- changed the license to be either the GPL or the Artistic license
- added Jay Sachs' implementation of NullCache
- modified the remove methods to avoid croaking if two cache
instances are both purging or limiting at the same time
- migrated to a factory-like model with private contructors
to fix the auto_purge_interval functionality
- updated the documentation to better reflect that size means
size in bytes
CVS tree (viewcvs):
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/perl-cache/
The following is the Cache-Cache-0.09 README file:
Copyright (C) 2001 DeWitt Clinton All Rights Reserved
You may distribute under the terms of either the GNU General Public
License or the Artistic License, as specified in the Perl README file.
NAME
Cache::Cache
DESCRIPTION
The Perl Cache package provides Cache::Cache, a generic interface
for creating persistent data stores. This interface is implemented
by the Cache::MemoryCache, Cache::SharedMemoryCache, Cache::FileCache,
Cache::SizeAwareFileCache, Cache::SizeAwareMemoryCache, and
Cache::SizeAwareSharedMemoryCache classes. This work replaces
File::Cache and IPC::Cache.
REQUIREMENTS
Digest::MD5
File::Spec
File::Path
IPC::ShareLite
Storable
INSTALLATION
perl Makefile.PL
make
make test
make install
USAGE
First, choose the best type of cache implementation for your needs.
The simplest cache is the MemoryCache, which is suitable for
applications that are serving multiple sequential requests, and
which to avoid making redundant expensive queries, such as an
Apache/mod_perl application talking to a database. If you wish to
share that data between processes, then perhaps the
SharedMemoryCache is appropriate, although its behavior is tightly
bound to the underlying IPC mechanism, which varies from system to
system, and is unsuitable for large objects or large numbers of
objects. When the SharedMemoryCache is not acceptable, then
FileCache offers all of the same functionality with similar
performance metrics, and it is not limited in terms of the number of
objects or their size. If you wish to maintain a strict limit on
the size of a file system based cache, then the SizeAwareFileCache
is the way to go. Similarly, the SizeAwareMemoryCache and the
SizeAwareSharedMemoryCache add size management functionality
to the MemoryCache and SharedMemoryCache classes respectively.
Using a cache is simple. Here is some sample code for instantiating
and using a MemoryCache:
use Cache::Cache qw( $EXPIRES_NEVER $EXPIRES_NOW );
use Cache::MemoryCache;
my $options_hash_ref = { 'default_expires_in' = '10 seconds' };
my $cache = new Cache::MemoryCache( $options_hash_ref );
my $expires_in = '10 minutes';
$cache-set( 'Key', 'Value', $expires_in );
# if the next line is called within 10 minutes, then this
# will return the cache value
my $value = $cache-get( 'Key' );
Please refer to the perldoc for Cache::Cache and the related
implementations for complete documentation.
SEE ALSO
File::Cache and IPC::Cache, and the project homepage at:
http://sourceforge.net/projects/perl-cache/
AUTHOR
Original author: DeWitt Clinton [EMAIL PROTECTED]
Last author: $Author: dclinton $
Copyright (C) 2001 DeWitt Clinton
[PRE-ANNOUNCE] Cache::Cache 0.09
Hi, If people have a moment, please check out Cache::Cache 0.09, which is available here until I am ready to release it on CPAN: http://prdownloads.sourceforge.net/perl-cache/Cache-Cache-0.09.tar.gz Changes include: - applied Axel Beckert patch to fix the expiration units - applied Ken Williams's directory creation patch to pass all tests - changed the license to be either the GPL or the Artistic license - added Jay Sachs' implementation of NullCache - modified the remove methods to avoid croaking if two cache instances are both purging or limiting at the same time - migrated to a factory-like model with private contructors to fix the auto_purge_interval functionality - updated the documentation to better reflect that size means size in bytes Cheers! -DeWitt
Re: Shared memory caching revisited (was it's supposed to SHARE it, not make more!)
On Tue, Sep 04, 2001 at 12:14:52PM -0700, Rob Bloodgood wrote: ***OH WOW!*** So, DURING the course of composing this message, I've realized that the function expire_old_accounts() is now redundant! Cache::Cache takes care of that, both with expires_in and max_size. I'm leaving it in for reference, just to show how it's improved. :-) [snip] use Apache::Session::Lock::Semaphore (); use Cache::SizeAwareSharedMemoryCache (); # this is used in %cache_options, as well as for locking use constant SIGNATURE = 'EXIT'; use constant MAX_ACCOUNTS = 300; # use vars qw/%ACCOUNTS/; use vars qw/$ACCOUNTS $locker/; my %cache_options = ( namespace = SIGNATURE, default_expires_in = max_size = MAX_ACCOUNTS ); Very neat thought about how to use max_size to limit the the accounts! Unfortunately, you demonstrated that I did a *terrible* job at documenting what size means. It means size in bytes, not items. I will add max_items and limit_items to the TODO list. In the meantime, I will improve the documentation. -DeWitt
Re: module to hit back at default.ida atack ?
On Tue, Aug 07, 2001 at 08:18:18PM +1000, Cees Hek wrote: So what your saying is that you have a list of potentially rooted machines that you are making publically available... Doesn't sound like such a good idea to me... So *that's* why Reuven has CodeRed.pm CC him on the warning emails. And I thought he was just nuts. ;) -DeWitt
Re: odd behavior with Cache::Cache
On Sat, Aug 04, 2001 at 09:17:21AM -0700, brian moseley wrote: could this be because i'm only ever doing one set, at session creation time? since i'm using a memory cache, i didn't expect to have to call set every time i modify the cached data. Yup. That would be it. Please see my response to my next post (the one in which you mention solving the problem). also i didn't know that Storable was used underneath. why is this necessary? To quote Jimi Williams, manager's decision. I decided to optimize for the more common scenario in which one is pulling objects out of the cache and modifying them. Rather than force the user to manually clone the retrieved object, I figured that it was better to do the clone up front and require the user to store the object again if they modify it. This is to preserve consistent semantics and behavior across the various cache implementations. I.e., what works for MemoryCache should work for the SharedMemoryCache or the FileCache as well. That's not to say one couldn't subclass MemoryCache to have the alternate behavior you propose... -DeWitt
Re: odd behavior with Cache::Cache
On Sat, Aug 04, 2001 at 10:34:34AM -0700, brian moseley wrote: ps: i've modified my code to 1) only get once per request and 2) set at the end of each request. the net effect is that stuff works as expected. i'm reasonably happy with the current state of affairs, but... Excellent, this is the right approach. Sounds like I need to update the documentation to say that objects retrieved from the cache are not 'live,' they are clones. If you want to save modifications, remember to store them again in the cache. i don't like having to explicitly call set to force modifications to be written to the cache, and i'd prefer that get always return the same instance that i originally set. can these issues be considered for a future version of the interface? i see the need for the current behavior when using file-based caches, but perhaps there's a way to streamline operations for memory caches? perhaps Storable can be bypassed for memory caches? Please see my last email on the subject, but for all intents and purposes, I'd like caches to behave consistently, but we could definitely create a special purpose live memory-based cache that does what you want. also, has there been any thought given to locking cached items? when i'm using a shared cache with multiprocess apache, the opportunity exists for multiple requests to access a single session simultaneously, which can lead to races. which i'm happy to ignore for now but would be nice to eventually prevent. A number of people have requested this, and I think it is a good idea. For some back-end implementations, locking is conceptually easy to do, such as the FileCache on a POSIX system. And the SharedMemoryCache already does some locking using IPC::ShareLite. However, I wanted to wait until I came up with the right level of granularity for the locking API. Plus, it would require a non-trivial bit of a rewrite to separate the front-end of the cache (which would block on locks) from the back-end (which would be aware of the locks and manage them). However, the good news is that there isn't really a race on writes. Basically, the last write wins. It is tough to really figure out (from the cache's perspective) what the appropriate behavior is in all cases. So the user should be responsible for locking if them want it. Of course, that should be done through the cache API, which I regretfully haven't added yet. Cheers, -DeWitt
Re: odd behavior with Cache::Cache
On Sat, Aug 04, 2001 at 03:06:37PM -0400, DeWitt Clinton wrote: To quote Jimi Williams, manager's decision. Yes, I know it is Jimy Williams. See what happens when a lifelong Red Sox fan relocates down to NYC? We start losing our minds... :) -DeWitt
[ANNOUNCE] Cache-Cache-0.08
Summary:
The Perl Cache package provides Cache::Cache, a generic interface
for creating persistent data stores. This interface is implemented
by the Cache::MemoryCache, Cache::SharedMemoryCache, Cache::FileCache,
Cache::SizeAwareFileCache, Cache::SizeAwareMemoryCache, and
Cache::SizeAwareSharedMemoryCache classes. This work replaces
File::Cache and IPC::Cache.
Release Notes:
This release adds the new auto purge feature, contains
minor API enhancements, and includes improved documentation.
Project Homepage:
http://sourceforge.net/projects/perl-cache/
Tar/GZ:
http://prdownloads.sourceforge.net/perl-cache/Cache-Cache-0.08.tar.gz
http://www.cpan.org/modules/by-module/Cache/Cache-Cache-0.08.tar.gz
Changelog:
http://sourceforge.net/project/shownotes.php?release_id=32744
CVS tree (viewcvs):
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/perl-cache/
The following is the Cache-Cache-0.08 README file:
Copyright (C) 2001 DeWitt Clinton All Rights Reserved
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 1, or (at your option)
any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
NAME
Cache::Cache
DESCRIPTION
The Perl Cache package provides Cache::Cache, a generic interface
for creating persistent data stores. This interface is implemented
by the Cache::MemoryCache, Cache::SharedMemoryCache, Cache::FileCache,
Cache::SizeAwareFileCache, Cache::SizeAwareMemoryCache, and
Cache::SizeAwareSharedMemoryCache classes. This work replaces
File::Cache and IPC::Cache.
REQUIREMENTS
Digest::MD5
File::Spec
File::Path
IPC::ShareLite
Storable
INSTALLATION
perl Makefile.PL
make
make test
make install
USAGE
First, choose the best type of cache implementation for your needs.
The simplest cache is the MemoryCache, which is suitable for
applications that are serving multiple sequential requests, and
which to avoid making redundant expensive queries, such as an
Apache/mod_perl application talking to a database. If you wish to
share that data between processes, then perhaps the
SharedMemoryCache is appropriate, although its behavior is tightly
bound to the underlying IPC mechanism, which varies from system to
system, and is unsuitable for large objects or large numbers of
objects. When the SharedMemoryCache is not acceptable, then
FileCache offers all of the same functionality with similar
performance metrics, and it is not limited in terms of the number of
objects or their size. If you wish to maintain a strict limit on
the size of a file system based cache, then the SizeAwareFileCache
is the way to go. Similarly, the SizeAwareMemoryCache and the
SizeAwareSharedMemoryCache add size management functionality
to the MemoryCache and SharedMemoryCache classes respectively.
Using a cache is simple. Here is some sample code for instantiating
and using a MemoryCache:
use Cache::Cache qw( $EXPIRES_NEVER $EXPIRES_NOW );
use Cache::MemoryCache;
my $options_hash_ref = { 'default_expires_in' = '10 seconds' };
my $cache = new Cache::MemoryCache( $options_hash_ref );
my $expires_in = '10 minutes';
$cache-set( 'Key', 'Value', $expires_in );
# if the next line is called within 10 minutes, then this
# will return the cache value
my $value = $cache-get( 'Key' );
Please refer to the perldoc for Cache::Cache and the related
implementations for complete documentation.
SEE ALSO
File::Cache and IPC::Cache, and the project homepage at:
http://sourceforge.net/projects/perl-cache/
AUTHOR
Original author: DeWitt Clinton [EMAIL PROTECTED]
Last author: $Author: dclinton $
Copyright (C) 2001 DeWitt Clinton
Re: Getting a Cache::SharedMemoryCache started
On Tue, Mar 27, 2001 at 09:50:33PM +0100, Matt wrote:
I want to cache some values in my handlers so they don't keep having
to look stuff up, and so I figured a Cache::SharedMemoryCache would
be the way to go.
It all depends on what you want to cache and why. If you wanted to
save a trip to the database on each request, a really simple mod_perl
handler could look something like this:
package MyApplication;
use strict;
use Apache::Request;
use Cache::SharedMemoryCache;
sub handler
{
my ( $request );
my $cache = new Cache::SharedMemoryCache( );
my $data = $cache-get( 'data' );
if ( not defined $data )
{
$data = expensive_database_request( );
$cache-set( 'data', $data );
}
$request-content_type( 'text/plain' );
$request-send_http_header( );
$request-print( "data: $data" );
}
The first time this handler is run, the data will not be found in the
cache. The second time, however, the data will be returned from the
cache, saving a trip to the database.
The other question is whether or not to share the cache instance
itself globally. Technically, this is up to you. Personally I
wouldn't bother, considering the overhead of instantiating the cache
is so low that I would rather keep it local to the handler (as I did
above). I tend to only share data globally in mod_perl handlers very
judiciously, and even then I wrap the data in an object that provides
accessor methods. For an example of an object that wraps shared data,
check out Cache::MemoryCache, which simply caches data inside the
process.
Hope this helps a bit. Cheers,
-DeWitt
Re: Getting a Cache::SharedMemoryCache started
On Tue, Mar 27, 2001 at 05:19:15PM -0500, Pierre Phaneuf wrote: DeWitt Clinton wrote: The other question is whether or not to share the cache instance itself globally. Technically, this is up to you. Personally I wouldn't bother, considering the overhead of instantiating the cache is so low that I would rather keep it local to the handler (as I did above). I tend to only share data globally in mod_perl handlers very judiciously, and even then I wrap the data in an object that provides accessor methods. For an example of an object that wraps shared data, check out Cache::MemoryCache, which simply caches data inside the process. Wouldn't this cause the cache to be torn down and rebuilt for every request??? The cache instance itself would be, but the data behind it (a global hash in the case of the MemoryCache, and a IPC::ShareLite store for the SharedMemoryCache ) would stay around. Since the instance isn't that heavy, I feel better instantiating them fresh every request. If you need optimal performance, however, you may want to keep the instance itself global. Which reminds me of something. These cache objects are not currently thread safe. When should I start expecting multi-threaded apache/mod_perl to become mainstream enough to warrant an overhaul of the code? I imagine that nearly most Perl libraries are not thread safe, of course. But code that will be used in mod_perl environments needs to be, right? -DeWitt
[ANNOUNCE] Cache-Cache-0.05
Summary:
The Perl Cache package provides Cache::Cache, a generic interface
for creating persistent data stores. This interface is implemented
by the Cache::MemoryCache, Cache::SharedMemoryCache, Cache::FileCache,
Cache::SizeAwareFileCache, Cache::SizeAwareMemoryCache, and
Cache::SizeAwareSharedMemoryCache classes. This work replaces
File::Cache and IPC::Cache.
Release Notes:
This release replaces IPC::Shareable with IPC::ShareLite as the
backend for the SharedMemoryCache and updates external module
dependencies to the latest versions.
Project Homepage:
http://sourceforge.net/projects/perl-cache/
Tar/GZ:
http://ftp1.sourceforge.net/perl-cache/Cache-Cache-0.05.tar.gz
Changelog:
http://sourceforge.net/project/shownotes.php?release_id=27936
CVS tree (cvsweb):
http://cvs.sourceforge.net/cgi-bin/cvsweb.cgi?cvsroot=perl-cache
The following is the Cache-Cache-0.05 README file:
Copyright (C) 2001 DeWitt Clinton All Rights Reserved
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 1, or (at your option)
any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
NAME
Cache::Cache
DESCRIPTION
The Perl Cache package provides Cache::Cache, a generic interface
for creating persistent data stores. This interface is implemented
by the Cache::MemoryCache, Cache::SharedMemoryCache, Cache::FileCache,
Cache::SizeAwareFileCache, Cache::SizeAwareMemoryCache, and
Cache::SizeAwareSharedMemoryCache classes. This work replaces
File::Cache and IPC::Cache.
REQUIREMENTS
Digest::MD5
File::Spec
File::Path
IPC::ShareLite
Storable
INSTALLATION
perl Makefile.PL
make
make test
make install
USAGE
First, choose the best type of cache implementation for your needs.
The simplest cache is the MemoryCache, which is suitable for
applications that are serving multiple sequential requests, and
which to avoid making redundant expensive queries, such as an
Apache/mod_perl application talking to a database. If you wish to
share that data between processes, then perhaps the
SharedMemoryCache is appropriate, although its behavior is tightly
bound to the underlying IPC mechanism, which varies from system to
system, and is unsuitable for large objects or large numbers of
objects. When the SharedMemoryCache is not acceptable, then
FileCache offers all of the same functionality with similar
performance metrics, and it is not limited in terms of the number of
objects or their size. If you wish to maintain a strict limit on
the size of a file system based cache, then the SizeAwareFileCache
is the way to go. Similarly, the SizeAwareMemoryCache and the
SizeAwareSharedMemoryCache add size management functionality
to the MemoryCache and SharedMemoryCache classes respectively.
Using a cache is simple. Here is some sample code for instantiating
and using a MemoryCache:
use Cache::Cache qw( $EXPIRES_NEVER $EXPIRES_NOW );
use Cache::MemoryCache;
my $options_hash_ref = { 'default_expires_in' = '10 seconds' };
my $cache = new Cache::MemoryCache( $options_hash_ref );
my $expires_in = '10 minutes';
$cache-set( 'Key', 'Value', $expires_in );
# if the next line is called within 10 minutes, then this
# will return the cache value
my $value = $cache-get( 'Key' );
Please refer to the perldoc for Cache::Cache and the related
implementations for complete documentation.
SEE ALSO
File::Cache and IPC::Cache
AUTHOR
Original author: DeWitt Clinton [EMAIL PROTECTED]
Last author: $Author: dclinton $
Copyright (C) 2001 DeWitt Clinton
[ANNOUNCE] Cache-Cache-0.04
Summary:
The Perl Cache package provides Cache::Cache, a generic interface
for creating persistent data stores. This interface is implemented
by the Cache::MemoryCache, Cache::SharedMemoryCache, Cache::FileCache,
Cache::SizeAwareFileCache, Cache::SizeAwareMemoryCache, and
Cache::SizeAwareSharedMemoryCache classes. This work replaces
File::Cache and IPC::Cache.
Release Notes:
This release offers two new classes, the SizeAwareMemoryCache
and the SizeAwareSharedMemoryCache, replaces the Data::Dumper
persistence mechanism with Storable, and adds a new
SizeAwareCache interface, plus some small bug fixes.
Project Homepage:
http://sourceforge.net/projects/perl-cache/
Tar/GZ:
http://ftp1.sourceforge.net/perl-cache/Cache-Cache-0.04.tar.gz
Changelog:
http://sourceforge.net/project/shownotes.php?release_id=27594
CVS tree (cvsweb):
http://cvs.sourceforge.net/cgi-bin/cvsweb.cgi?cvsroot=perl-cache
The following is the Cache-Cache-0.04 README file:
Copyright (C) 2001 DeWitt Clinton All Rights Reserved
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 1, or (at your option)
any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
NAME
Cache::Cache
DESCRIPTION
The Perl Cache package provides Cache::Cache, a generic interface
for creating persistent data stores. This interface is implemented
by the Cache::MemoryCache, Cache::SharedMemoryCache, Cache::FileCache,
Cache::SizeAwareFileCache, Cache::SizeAwareMemoryCache, and
Cache::SizeAwareSharedMemoryCache classes. This work replaces
File::Cache and IPC::Cache.
REQUIREMENTS
Digest::MD5
File::Spec
File::Path
IPC::Shareable
Storable
INSTALLATION
perl Makefile.PL
make
make test
make install
USAGE
First, choose the best type of cache implementation for your needs.
The simplest cache is the MemoryCache, which is suitable for
applications that are serving multiple sequential requests, and
which to avoid making redundant expensive queries, such as an
Apache/mod_perl application talking to a database. If you wish to
share that data between processes, then perhaps the
SharedMemoryCache is appropriate, although its behavior is tightly
bound to the underlying IPC mechanism, which varies from system to
system, and is unsuitable for large objects or large numbers of
objects. When the SharedMemoryCache is not acceptable, then
FileCache offers all of the same functionality with similar
performance metrics, and it is not limited in terms of the number of
objects or their size. If you wish to maintain a strict limit on
the size of a file system based cache, then the SizeAwareFileCache
is the way to go. Similarly, the SizeAwareMemoryCache and the
SizeAwareSharedMemoryCache add size management functionality
to the MemoryCache and SharedMemoryCache classes respectively.
Using a cache is simple. Here is some sample code for instantiating
and using a MemoryCache:
use Cache::Cache qw( $EXPIRES_NEVER $EXPIRES_NOW );
use Cache::MemoryCache;
my $options_hash_ref = { 'default_expires_in' = '10 seconds' };
my $cache = new Cache::MemoryCache( $options_hash_ref );
my $expires_in = '10 minutes';
$cache-set( 'Key', 'Value', $expires_in );
# if the next line is called within 10 minutes, then this
# will return the cache value
my $value = $cache-get( 'Key' );
Please refer to the perldoc for Cache::Cache and the related
implementations for complete documentation.
SEE ALSO
File::Cache and IPC::Cache
AUTHOR
Original author: DeWitt Clinton [EMAIL PROTECTED]
Last author: $Author: dclinton $
Copyright (C) 2001 DeWitt Clinton
[OT] Re: mod_perl shared memory with MM
On Sun, Mar 11, 2001 at 03:33:12PM +0100, Christian Jaeger wrote:
I've looked at Cache::FileCache now and think it's (currently) not
possible to use for IPC::FsSharevars:
I really miss locking capabilities. Imagine a script that reads a
value at the beginning of a request and writes it back at the end of
the request. If it's not locked during this time, another instance
can read the same value and then write another change back which is
then overwritten by the first instance.
I'm very intrigued by your thinking on locking. I had never
considered the transaction based approach to caching you are referring
to. I'll take this up privately with you, because we've strayed far
off the mod_perl topic, although I find it fascinating.
- why don't you use 'real' constants for $SUCCESS and the like? (use
constant)
Two reasons, mostly historical, and not necessarily good ones.
One, I benchmarked some code once that required high performance, and
the use of constants was just slightly slower.
Two, I like the syntax $hash{$CONSTANT}. If I remember correctly,
$hash{CONSTANT} didn't work. This may have changed in newer versions
of Perl.
Obviously those are *very* small issues, and so it is mostly by habit
that I don't use constant. I would consider changing, but it would
mean asking everyone using the code to change too, because they
currently import and use the constants as Exported scalars.
Do you know of a very important reason to break compatibility and
force the switch? I'm not opposed to switching if I have to, but I'd
like to minimize the impact on the users.
- you probably should either append the userid of the process to
/tmp/FileCache or make this folder globally writeable (and set the
sticky flag). Otherwise other users get a permission error.
As of version 0.03, the cache directories, but not the cache entries,
are globally writable by default. Users can override this by changing
the 'directory_umask' option, or keep data private altogether by
changing the 'cache_root'. What version did you test with? There may
be a bug in there.
- why don't you use Storable.pm? It should be much faster than Data::Dumper
The TODO contains "Replace Data::Dumper with Storable (maybe)". :)
The old File::Cache module used Storable, btw.
It will be trivial to port the new Cache::FileCache to use Storable.
I simply wanted to wait until I had the benchmarking code so I could
be sure that Storeable was faster. Actually, I'm not 100% sure that I
expect Storeable to be faster than Data::Dumper. If Data::Dumper
turns out to be about equally fast, then I'll stay with it, because it
is available on all Perl installations, I believe.
Do you know if Storeable is definitely faster? If you have benchmarks
then I am more than happy to switch now. Or, do you know of a reason,
feature wise, that I should switch? Again, it is trivial to do so.
I have some preliminary benchmark code -- only good for relative
benchmarking, but it is a start. I'd be happy to post the results
here if people are interested.
Could you send me the code?, then I'll look into benchmarking my
module too.
I checked it in as Cache::CacheBenchmark. It isn't good code, nor
does it necessarily work just yet. I simply checked it in while I was
in the middle of working on it. I'm turning it into a real
benchmarking class for the cache, and hopefully that will help you a
little bit.
Cheers,
-DeWitt
[ANNOUNCE] Cache-Cache-0.03
Summary:
Perl Cache is the successor to the popular File::Cache and
IPC::Cache perl libraries. This project unifies those modules under
the generic Cache::Cache interface and implements Cache::FileCache,
Cache::MemoryCache, Cache::SharedMemoryCache, and
Cache::SizeAwareFileCache.
Release Notes:
This release offers performance enhancements, expiration and size
reduction bug fixes, an improved expiration string syntax, much
better documentation, and a new cache API method.
Project Homepage:
http://sourceforge.net/projects/perl-cache/
Tar/GZ:
http://ftp1.sourceforge.net/perl-cache/Cache-Cache-0.03.tar.gz
Changelog:
http://sourceforge.net/project/shownotes.php?release_id=26779
CVS tree (cvsweb):
http://cvs.sourceforge.net/cgi-bin/cvsweb.cgi?cvsroot=perl-cache
The following is the Cache-Cache-0.03 README file:
Copyright (C) 2001 DeWitt Clinton All Rights Reserved
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 1, or (at your option)
any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
NAME
Cache::Cache
DESCRIPTION
The Perl Cache package provides Cache::Cache, a generic interface
for creating persistent data stores. This interface is implemented
by the Cache::FileCache, Cache::SizeAwareFileCache,
Cache::MemoryCache, and Cache::SharedMemoryCache classes. This work
replaces File::Cache and IPC::Cache.
REQUIREMENTS
Data::Dumper
Digest::MD5
File::Spec
File::Path
IPC::Shareable
INSTALLATION
perl Makefile.PL
make
make test
make install
USAGE
First, choose the best type of cache implementation for your needs.
The simplest cache is the MemoryCache, which is suitable for
applications that are serving multiple sequential requests, and
which to avoid making redundant expensive queries, such as an
Apache/mod_perl application talking to a database. If you wish to
share that data between processes, then perhaps the
SharedMemoryCache is appropriate, although its behavior is tightly
bound to the underlying IPC mechanism, which varies from system to
system, and is unsuitable for large objects or large numbers of
objects. When the SharedMemoryCache is not acceptable, then
FileCache offers all of the same functionality with similar
performance metrics, and it is not limited in terms of the number of
objects or their size. If you wish to maintain a strict limit on
the size of a file system based cache, then the SizeAwareFileCache
is the way to go.
Using a cache is simple. Here is some sample code for instantiating
and using a MemoryCache:
use Cache::Cache qw( $EXPIRES_NEVER $EXPIRES_NOW );
use Cache::MemoryCache;
my $options_hash_ref = { 'default_expires_in' = '10 seconds' };
my $cache = new Cache::MemoryCache( $options_hash_ref );
my $expires_in = '10 minutes';
$cache-set( 'Key', 'Value', $expires_in );
# if the next line is called within 10 minutes, then this
# will return the cache value
my $value = $cache-get( 'Key' );
Please refer to the perldoc for Cache::Cache and the related
implementations for complete documention.
SEE ALSO
File::Cache and IPC::Cache
AUTHOR
Original author: DeWitt Clinton [EMAIL PROTECTED]
Last author: $Author: dclinton $
Copyright (C) 2001 DeWitt Clinton
Re: [ANNOUNCE] Cache-Cache-0.03
On Sat, Mar 10, 2001 at 11:17:21AM -0800, Bill Moseley wrote: When you say successor to File::Cache does that means File::Cache will not be maintained as a separate module anymore? There are no features in File::Cache that will not exist in Cache::FileCache and Cache::SizeAwareFileCache. Actually, as far as I know, the only old feature missing at all from the new code is the ability to tell the cache not to physically remove expired objects. There is a backdoor around this in 0.03, however, so if you need that functionality, it is already possible. I'm hoping to retire File::Cache soon unless I get another maintainer. But it seems silly to have redundant efforts. The new code is much, much, better. Have you though about making SharedMemoryCache flush to disk if it becomes full but before it's time to expire the data? I've done a lot of thinking about a multi-layered cache implementation. The API would be the same, but it would be clever about using MemoryCache - SharedMemoryCache - FileCache to make read access to data super efficient, and only persist outward on writes. I'll mark that down as a potential 0.04 feature. Thanks, Bill! -DeWitt
Re: [ANNOUNCE] Cache-Cache-0.03
On Sat, Mar 10, 2001 at 02:00:34PM -0600, Daniel Little (Metrex) wrote: Along the same lines, how about making SizeAwareMemoryCache as well so that you can specify just how much data you want stored in the cache. Good idea. I'll add that to the 0.04 feature list as well. Thanks! -DeWitt
Re: [ANNOUNCE] Cache-Cache-0.03
On Sat, Mar 10, 2001 at 12:10:35PM -0800, Perrin Harkins wrote: "Daniel Little (Metrex)" wrote: Along the same lines, how about making SizeAwareMemoryCache as well so that you can specify just how much data you want stored in the cache. Sounds like Joshua Chamas' Tie::Cache module. It provides a size-limited LRU cache. Note that Cache::SizeAwareFileCache does both a "next expiration" and a LRU algorithm for limiting the size. I'll use the same logic for the SizeAwareMemoryCache. Thanks, -DeWitt
Re: mod_perl shared memory with MM
On Sat, Mar 10, 2001 at 04:35:02PM -0800, Perrin Harkins wrote: Christian Jaeger wrote: Yes, it uses a separate file for each variable. This way also locking is solved, each variable has it's own file lock. You should take a look at DeWitt Clinton's Cache::FileCache module, announced on this list. It might make sense to merge your work into that module, which is the next generation of the popular File::Cache module. Yes! I'm actively looking for additional developers for the Perl Cache project. I'd love new implementations of the Cache interface. Cache::BerkeleyDBCache would be wonderful. Check out: http://sourceforge.net/projects/perl-cache/ For what it is worth, I don't explicitly lock. I do atomic writes instead, and have yet to hear anyone report a problem in the year the code has been public. It's a bit difficult to write a realworld benchmark. It certainly is. Benchmarking all of the options is something that I've always wanted to do and never find enough time for. I have some preliminary benchmark code -- only good for relative benchmarking, but it is a start. I'd be happy to post the results here if people are interested. -DeWitt
[ANNOUNCE] Cache-Cache-0.02
Summary: Perl Cache is the successor to the popular File::Cache and IPC::Cache perl libraries. This project unifies those modules under the generic Cache::Cache interface and implements Cache::FileCache, Cache::MemoryCache, Cache::SharedMemoryCache, and Cache::SizeAwareFileCache. Release Notes: This release implements Cache::SizeAwareFileCache, which extends Cache::FileCache and adds dynamic cache sizing functionality Project Homepage: http://sourceforge.net/projects/perl-cache/ Tar/GZ: http://download.sourceforge.net/perl-cache/Cache-Cache-0.02.tar.gz Changelog: http://sourceforge.net/project/shownotes.php?release_id=24111 CVS tree (cvsweb): http://cvs.sourceforge.net/cgi-bin/cvsweb.cgi?cvsroot=perl-cache
Re: [OT] re:advocacy at perl.com
On Fri, Feb 16, 2001 at 03:14:37PM +, Matt Sergeant wrote: I wonder if I should write them an article about take23, and then use the funds to pay for articles for take23 :-) Geez, Matt, you promote take23 every chance you get. That's worse than me mentioning Avacet (www.avacet.com), Perl Cache (sourceforge.net/projects/perl-cache), and the fact that I'm looking for a job (www.unto.net/resume.html). ;) -DeWitt
[ANNOUNCE] Cache::Cache 0.01
Hello: Perl Cache is the successor to the popular File::Cache and IPC::Cache perl libraries. This project unifies those modules under the generic Cache::Cache interface and implements Cache::FileCache, Cache::MemoryCache, and Cache::SharedMemoryCache. The project is being hosted on SourceForge at: http://sourceforge.net/projects/perl-cache/ I have released the first version of the unified Cache code as version 0.01, and it can be downloaded here: http://download.sourceforge.net/perl-cache/Cache-Cache-0.01.tar.gz I have not implemented the full functionality of File::Cache (e.g., dynamic cache sizing) quite yet. I am in the process of porting that code and it will be included in the next release as a new module. Comments and critique always welcome! Cheers, -DeWitt
Re: File::Cache problem
On Wed, Feb 07, 2001 at 04:02:39PM +0400, BeerBong wrote:
Cache size after 24 hours of working
via 'du -s' - 53M
via 'perl -MFile::Cacje -e"print File::Cache::SIZE('PRTL')"' - 10M
That looks like a serious bug. Can you double check on that? Where
are you running the "du -s"?
The SIZE( ) method returns the total size of the objects in the cache,
not including the overhead of directories themselves. As the
documentation explains, it isn't a perfect mechanism, but it should be
a lot closer than you experienced.
What is the best way to limit cache ?
May be variant when File::Cache::REDUCE_SIZE(10485760) in cron.dayly
will be better solution ? I this case max_size doesnt need to specify
in object initialization params...
I personally prefer using a cron job to periodically limit the size of
the cache. I don't like using "max_size" because of the overhead
involved on every set. I'd rather just fine tune the number of times
a cron script is executed (i.e., once an hour vs. once a day).
Another kind of problem appears recently.
In logs
[Wed Feb 7 15:42:57 2001] [error] Couldn't rename
/tmp/File::Cache/.description.tmp13984 to /tmp/File::Cache/.description at
/usr/web/inc/WebCache.pm line 15
[Wed Feb 7 15:44:37 2001] [error] Couldn't rename
/tmp/File::Cache/apache/PRTL/b/5/a/b5a261aa13cf0ddfd41beb7d64960248.tmp13441 to
/tmp/File::Cache/apache/PRTL/b/5/a/b5a261aa13cf0ddfd41beb7d64960248 at
/usr/web/inc/WebCache.pm line 87
...propagated at /usr/web/inc/Apache/Portal.pm line 109.
What I can do ?
Unfortunately the error message doesn't tell me enough. If you can,
please modify lines 1578 and 1579 of Cache.pm to read as follows:
rename ($temp_filename, $filename) or
croak("Couldn't rename $temp_filename to $filename: $!");
The next time it hangs, the error message will have more information.
I'll fix that in the next version of File::Cache.
And another question (I'm use Linux Debian Potato)...
Is there way to define params of the currently performing request
(URI, script, etc) of hanging mod_perl process ? I browsed /proc/[PID of
mod_perl]/, but I think this info is about first request. Am I wrong ?
Are you looking in /proc/[pid]/environ for the request parameters?
I'm surprised this information was available even for the first
request. I don't believe environ is updated dynamically (I think it
is filled out once on the exec of the process). It certainly won't
provide the information on subsequent requests. In any case, the
format and behavior of /proc is so dependent on the particular version
of the Linux kernel that I wouldn't want to rely on it for anything
more than occasional debugging.
Two options for you would be to log the request every time via warn.
It will fill your logs very quickly, but hopefully it will hang
quickly and you can get the debugging information out.
Another solution is to compile apache and mod_perl with debugging
symbols and attach to it with "gdb". I have tried this a long time,
though. Please see the current thread about using gdb.
Best of luck!
-DeWitt
Re: pseudo-hashes? was: Data structure question
On Tue, Jan 23, 2001 at 10:06:13AM +, Matt Sergeant wrote: The only gain might be in a large DOM tree where there may be thousands of objects. But then you're really better off using an array based class instead (as I found out). This is getting a bit off-topic, but I'm empirically found that the DOM is not necessarily the best object model to use in a mod_perl environment. XML::DOM in particular has such a high overhead in terms of memory (and memory leaks) and performance, that it is sometimes inappropriate for a context that requires a small footprint, and generally fast throughput (like mod_perl). For example, in version 1 of the Avacet perl libraries, we were using XML::DOM for both our XML-RPC mechanism and as the underlying data structure for object manipulation. In version 2, however, we created an architecture that automatically converts between the language agnostic XML and native blessed objects using a custom engine built on the PerlSAX parser. This reduced our memory footprint dramatically, stopped up the memory leaks, and increased performance significantly. Moreover, the object model now exposed is based on native perl objects with an API geared toward property manipulation (i.e., get_foo, set_foo) which is easier to program directly to than the DOM. You can see this in action with the modules available in the Avacet::Core::Rpc::Xml namespace at www.avacet.com. Best regards, -DeWitt
Re: Caching search results
On Mon, Jan 08, 2001 at 10:10:25AM -0800, Perrin Harkins wrote: Always start with CPAN. Try Tie::FileLRUCache or File::Cache for starters. A dbm would be fine too, but more trouble to purge old entries from. If you find that File::Cache works for you, then you may also want to check out the simplified and improved version in the Avacet code, which additionally offers a unified service model for mod_perl applications. Services are available for templates (either Embperl or Template Toolkit), XML-based configuratio, object caching, connecting to the Avacet application engine, standardized error handling, dynamically dispatching requests to modules, and many other things. -DeWitt
[ANNOUNCE] Avacet Application Engine 2.0b2
Hi! I've been quiet for a while on this list, but I still read it religiously. I wanted to take a second and share our announcement of the release of the second beta of the Avacet Application Engine. Version 2.0b2 of the Avacet Application Engine is available for download at http://www.avacet.com. The Avacet software suite consists of two main parts. The first is a language-neutral definition of data and interfaces for exchanging data, and the second is a reference implementation of an application engine that implements the Avacet framework and client libraries. It is suitable for the development of custom enterprise applications that require a high degree of scalability, reliability, and interoperability. The second beta release of the Avacet Application Engine adds a language-independent definition format for module interfaces, offers XML/RPC over HTTP, new HOWTO documentation geared toward new users, and many other enhancements. This is the last beta before version 2.0 of the Avacet Application Engine Perhaps more so than other application platforms, the Avacet engine is targeted toward building enterprise quality applications in a Perl and Apache/mod_perl environment. The Avacet framework itself is a language-neutral and open standard for describing data and protocols for data exchange. Because the definition is not represented in a particular programming language, we are able to implement client libraries in multiple languages (such as Perl) that can share common middle-tier business logic. I fundamentally believe that Perl, when written well, can be an enterprise ready programming language, and that Apache/mod_perl is one of the best environments (if not the best) for building web applications. To that end, we have built a feature rich set of modules that ease the development of mod_perl applications. The Avacet Perl libraries offer a ModPerlApplication base class, as well as a Service object model for dynamic templates (using HTML::Embperl), XML based configuration, memory and file system based object caching, an "exception" model for Perl, among many other features. I'm announcing this on the mod_perl list because I hope that our code offers value to engineers doing mod_perl development. We are using the Avacet engine and the Perl modules for our own e-commerce application suite. By taking advantage of the Avacet framework, business logic can be shared between custom mod_perl apps, those we develop at Avacet, and those developed at large by the community. We have mailing lists set up at the Avacet site, and I invite people to join us at the avacet-developers list. I'd be happy to set up an avacet-perl list if the mod_perl community finds these libraries to be useful. You are all invited to check out the second avacet beta at http://www.avacet.com. I look forward to hearing your feedback! Best regards, -DeWitt
[OT ANNOUNCE] Beans-0.01
Hi,
This may be one of the strangest programming things I've ever tried to
do. Because I've recently been introduced (somewhat embarrassingly)
to the joys of Java Beans, I thought "hey, wouldn't this be great to
have in Perl, too?"
Well, since I'm masochistic like that, I've written a very partial
implementation of the Java Beans framework in Perl.
It only supports properties for the most part, so it is hardly ready
for serious projects. But the framework is in place, and over time
this may actually turn out to have been a good use of time.
I'm including the README in this mail. You can grab the 0.01 version
of Perl Beans at the following location:
http://unto.net/archives/Beans-0.01.tar.gz
If anyone out there in the world thinks this idea has any merit
whatsoever, I'll probably create a SourceForge project for it and
submit it to CPAN. As always, feedback, advice, and criticism are
always welcome.
-DeWitt
README:
Copyright (C) 2000 DeWitt Clinton [EMAIL PROTECTED], Avacet, Inc.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 1, or (at your option)
any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
NAME
Beans::*
DESCRIPTION
Beans is a (very) partial implementation of the Java Beans framework
INSTALLATION
perl Makefile.PL
make
make test
make install
NOTES
The implementation is not even close to being complete. While a
number of class stubs have been provided, most of the interesting
parts are still not functional.
In particular, the entire event model is absent, along with the
persistence mechanisms. Additionally, since this is at proof of
concept stage, not effort has been put into making these classes
performant. Numerous caching techniques could be put in place to
speed things up.
However, over time there is a reasonable chance that the missing
pieces are filled in. Feel free to contribute!
USAGE
A very simple example is as follows (please excuse the formatting):
First, define a class that has methods getFoo and setFoo in
accordance with the Beans convention.
package GenericBean;
sub new {
my ($proto) = @_;
my $class = ref($proto) || $proto;
my $self = {};
bless ($self, $class);
return $self;
}
sub getFoo {
my ($self) = @_;
return $self-{_foo};
}
sub setFoo {
my ($self, $foo) = @_;
$self-{_foo} = $foo;
}
sub getBar {
my ($self) = @_;
return $self-{_bar};
}
sub setBar {
my ($self, $bar) = @_;
$self-{_bar} = $bar;
}
First, create an instance of the GenericBean class:
use GenericBean;
my $genericBean = new GenericBean();
Now, you can use the Beans classes to reflect on a run-time instance
of that class. Since Perl doesn't have exactly the same notion of
Class that Java does, the semantics are a little different.
use Beans::Class;
use Beans::Introspector;
use Beans::BeanInfo;
use Beans::PropertyDescriptor;
use Beans::Method;
my $genericBeanClass =
Beans::Class::getClassFromInstance($genericBean);
my $beanInfo =
Beans::Introspector::getBeanInfo($genericBeanClass);
my @propertyDescriptors = $beanInfo-getPropertyDescriptors();
You can iterate over the property descriptors and actually write
values to each property:
my $i;
foreach my $propertyDescriptor (@propertyDescriptors) {
my $writeMethod = $propertyDescriptor-getWriteMethod();
$writeMethod-invoke($genericBean, $i);
$i++;
}
Or, read those values back out again:
foreach my $propertyDescriptor (@propertyDescriptors) {
my $readMethod = $propertyDescriptor-getReadMethod();
my $propertyValue = $readMethod-invoke($genericBean);
my $propertyName = $propertyDescriptor-getName();
print "$propertyName: $propertyValue\n";
}
SEE ALSO
Sun's Java Beans tutorial:
http://www.javasoft.com/docs/books/tutorial/javabeans/index.html
Sun's JDK 1.3 API documentation for the java.beans package:
http://java.sun.com/j2se/1.3/docs/api/java/beans/package-summary.html
Re: [OT] Bean::*?
On Wed, May 31, 2000 at 04:16:09PM -0700, Perrin Harkins wrote: Maybe I'm just being thick, but if you leave out the event driven parts of the spec, what's left that isn't just an ordinary Perl class? You can easily make classes to hold data using Class::Struct or Class::MethodMaker. In all this time, I don't think I've seen you simply being thick even once. :) I agree, if you take out the event driven stuff, beans aren't that exciting. They are basically just property bags with a way to reflect over those properties. However, if I build the beginning of the bean model in Perl, then someone might come along and add the event model. Is it just that you want a safer way to do introspection than looking at the data structure the class is made from? Maybe you want the Class::Fields module from CPAN. Trust me -- this has nothing to do with "safe." :) The bean model is simply a convention -- if a class has a getFoo and a setFoo, then it has the property Foo. Like I said, I don't want to debate whether or not it is a good standard. The entire point is to have a consistent framework between Java and Perl. Chances are, the underlying mechanism for doing the reflection will leverage something from the Class:: package. There are a million ways I'd rather see this solved in Perl. Or in Java for that matter. But java.beans are pretty popular, and they provide the underpinnings for EJB, which, for better or for worse, is very prevalent in e-commerce. I'd rather support a mediocre standard then invent a proprietary one. And beans aren't *that* mediocre. -DeWitt
[OT] Bean::*?
Hi all, Okay, this is a rather ridiculous question. I spent the weekend implementing a property based object model in Java. However, after I finished, someone in my company laughingly pointed out that I had just re-invented Java beans. While there were some minor advantages to the model I came up with, the overwhelmingly richer feature set of the java.beans.* framework made my efforts redundant at best. One of the advantages, however, is that the object model I was working on would translate very nicely to Perl. In fact, the language independent nature of this (other than OO being a requirement) was the real reason I bothered with it at all. My question is this -- has anyone written an implementation of the Java Bean standard in Perl? I don't think we need to have a major debate about the shortcomings of beans on the mod_perl list. But I am wondering if anyone has pulled this off yet. If we left out the event driven parts of the bean spec, and just focused on the property side of things, this could be a rather short project to get the core down. I'd like to see the Perl version be API compatable with the Java version. There is no need to take it much further. Thoughts? Flames? :) -DeWitt
Re: mod_perl and IPC
On Mon, May 22, 2000 at 12:06:20PM -0700, Karyn Ulriksen wrote: IPC::Cache has worked like a champ for me on a heavily loaded webserver. It's simple and (I'm lead to believe) it's a compiled module. You interact with it like a hash file, but if you're clever you can work it out for your usage without semaphores. It also behaves itself very nicely (no memory leaks that I have run into). Hi, I'm the author of IPC::Cache and frankly, I'm a little surprised it's worked so well for you. :) I've noticed some performance issues with it. The good news, however, is that Sam Tregar, author of IPC::SharedCache, and I have been working on a major upgrade. In the next release, IPC::Cache will be back-ended against IPC::SharedCache module. It will now scale and perform much better. The other alternative is to use File::Cache (also available on CPAN). File::Cache actually has better performance characteristics than IPC::Cache. Since switching between the two usually entails simply replacing the string "IPC::Cache" with "File::Cache" it may be worth checking out. I could use a few beta testers of the new IPC::Cache module. If anyone wants to help, please let me know. -DeWitt
Re: mod_perl and IPC
On Mon, May 22, 2000 at 04:35:54PM -0400, Drew Taylor wrote: Could I ask what the problems you were having? I can't use IPC::SharedCache currently because I get the error "Couldn't find free segment" or something like that when trying to initialize the cache. The system engineers are supposed to be changing this kernel parameter, but I haven't heard back from them yet. The problem had to do with large numbers of objects in the cache. Rather than rephrase what Sam wrote to me, I'll just excerpt it here: In case you're wondering why I'm saying IPC::Cache needs help, I remember discussing two points. First, it's storing every cached object in the same giant IPC segment and copying it into local memory on every cache access. This essentially means that you are gaining *no* memory-usage advantage by using shared memory. Also, it wastes a lot of time copying large hunks of memory on every access. Second, you're doing reads inside an exclusive lock. This means that no two programs can be reading at the same time. This wastes the potential for efficient multi-processing with shared memory. He was absolutely correct about this. Since he is actively maintaining and improving IPC::SharedCache, I figured it makes sense to simply provide the straightforward IPC::Cache style interface around his code. Right now, things are in a holding pattern because I'm finding a limit on the number of objects I can put in the cache (less than 100, so it is an issue). Hopefully Sam will offer some insight here. -DeWitt
Re: mod_perl and IPC
On Mon, May 22, 2000 at 03:46:41PM -0700, Karyn Ulriksen wrote: I know what beta testing is, so I hope you didn't think I was trying to yank your chain! I knew that if it got ugly I could easily take that server offline and easily back out the code. Ack! I'm sorry for what I think is a miscommunication. I think the cordial tone was lost in e-mail. :) I was being sincere, I definitely appreciate your testing the code. As it was, it got even uglier since I last wrote in. Again this is FYI... for input on the beta and all... The load balancing service I'm using masked the more serious problem of the data failing with get passed somewhere in the bowels of Apache/mod_perl where it was before. I'll tinker with it in the wee hours to see if it is failing on the set or the retrieve. I reviewed the syntax and there doesn't appear to be any changes between new and old (did I miss anything?... it's a little late in the day and I had a long weekend :) and it's possible I did and I didn't review for any caveats... ). I'm pretty sure you are doing exactly the right thing. It is certainly possible that the new version doesn't work at all. I'd love to figure out a way to make this version work, because it does address some of the concerns that Sam pointed out. However, if there is degraded performance (which may be an understatement) then I won't release version 0.04 like this. Again, thanks for the highly valuable feedback! -DeWitt
Re: Implementing security in CGI
Hi, Interesting thread and interesting question. It makes sense to start with the requirements for what it means to implement those secure features. My requirements have an obvious e-commerce bias, and should probably be heavily reviewed by anyone thinking of using this design for online banking or government work. First, I'd like to introduce the notion of a secure session. Secure sessions have some subtle differences from the traditional notion of the session, and I'll point those out when they appear. The secure session has the following properties: *) The user is able to initiate a secure session by providing proper credentials (i.e., a username and password pair) via a login process. *) The user is able to terminate the secure session via a logout process. *) Secure sessions must be able to time out automatically. *) Secure sessions must *never* transmit sensitive data (such as the password) over insecure channels. *) The secure session, while it requires the use of a secure protocol (such as HTTPS), should not require the use of cookies. Cookies, however, can be employed to extend the functionality of the system. Additionally, I feel that one of the essential requirements to any enterprise quality, highly scalable, and fault-tolerant system is the ability to store session state on a tier other than the front-end. This usually means storing state in a shared database, but there are other options. A very effective method of meeting the requirements above is to use a two token architecture. The first token is a user identifier, which can be passed over insecure channels. This user identification token can be used to restore state that isn't particularly sensitive, such as a shopping cart or user preferences. The second token is a secure identifier, which is never passed over insecure channels. The secure identifier is required to access sensitive data, such as credit card data. (It is possible to create an architecture that uses *only* the secure token, but there are significant benefits in terms of flexibility afforded by using two tokens, so I won't go into the one token model here.) The fundamental goal of the secure token is to a) keep it safe from prying eyes, and b) use it is a requirement for accessing secure data. Tokens can be passed in one of two ways. First, the token can be passed as part of the URL. Second, the token can be passed in a cookie. I've found it very useful to create an initialization procedure inside applications that check both places for these tokens, and abstract the particular mechanism used to pass them. Note, however, that is important to remember how the token was passed -- if it came from a cookie, it is cosmetic to not pass it around in the URL. I also recommend creating a library used in your templates that has a function to build URLs that automatically append the tokens if necessary. Since it is critical that secure tokens are never passed in the clear, it really helps to have that function, because it can prevent the secure token from being sent via an insecure page in a URL (i.e., only setting secure tokens in URLs that contain "https"). The user identification token is typically used to restore state from the database. It is created whenever a page is viewed and the client has either not provided one (via the URL or a cookie), or the token they provide doesn't exist in the database. This token should then be set in both the URL and in a cookie. On the subsequent page request, if the user identification token is found in a cookie, it is probably safe to stop putting it in the URL. However, if the token is found only in the URL (probably meaning the client does not use cookies) the server should not try writing the cookie again. The advantage of the cookie is such that the user can insecurely identify themselves across browser sessions. This is perfectly acceptable when used to restore some insensitive state data (such as a name or shopping cart). However, the system still functions without the cookie. The user can now login to associate their client with particular stored user information. The login process is as follows: 1) The client connects to a secure page that presents a form requesting a username and password. (There has been much discussion about whether a HTTP form that POSTS to a secure page will encrypt the posted data. There is too much ambiguity here -- I recommend using HTTPS on the login form as well.) The ACTION of the form must also be a secure page. 2) On receiving the username and password, the server compares an encrypted version of the password with the stored encrypted password associated with that customer. If the passwords do not match, the user is returned to the login page. (For highly secure sites, storing a count of the number of failed logins may be desirable.) 3) The server compares the client supplied user identification token with the one associated with that username in
Re: Caching DB queries amongst multiple httpd child processes
On Thu, 3 Feb 2000, Peter Skipworth wrote: Does anyone have any experience in using IPC shared memory or similar in caching data amongst multiple httpd daemons ? Well, I just released IPC::Cache version 0.02 to CPAN. Two major caveats, however -- first, it is not currently in production on our site (www.eziba.com) and won't be for another week or two. Second, it relies on IPC::ShareLite as a backend. However, it does offer a very straightforward interface, and attempts to manage the memory segments and clean up after them. We are running Linux 2.2 on the front end, btw, and that may account for why ShareLite seems rather reliable for us. I'd love it if you could download it and let me know it works for you. We are planning on maintaining this, and would appreciate any feedback you can give about how to improve it. You can get IPC::Cache here: http://www.cpan.org/modules/by-module/IPC/IPC-Cache-0.02.tar.gz -DeWitt
Version 0.02 of IPC::Cache available
Hi, [I sent this out a while back, but it never made it through. Odd.] Based on some great feedback from this list, I was able to remove the dependency on IPC::Shareable. Not that I dislike IPC::Shareable -- to the contrary, it does what it does rather well -- but I was really looking for something more lightweight. I discovered Maurice Aubrey's excellent IPC::ShareLite module, which exactly filled the role I needed. IPC::Cache now relies on the efficient ShareLite shared memory routines, but the interface did not change at all. You can find it at CPAN now: http://www.cpan.org/modules/by-module/IPC/IPC-Cache-0.02.tar.gz Thanks for the feedback! -DeWitt
IPC::Cache version 0.01 available
Hi, A week or two ago, in the squid performance thread, I mentioned that I was looking for ways to eliminate squid from our production servers. I noted that we are using squid to save an expensive trip to the database to retrieve mostly static files. At that time I said that I planned to write a simple cache using shared memory that could be used with mod_perl to keep data around on the first tier. This weekend I finally had an opportunity to do just that. IPC::Cache is rather simple, considering that all the hard work is being done in the IPC::Shareable the Storable modules. I just added some basic namespace semantics and the ability to have data expire after a period of time. I'm planning on submitting it to CPAN, but I think it makes sense to make it available on this list before I do. I'd appreciate it if anyone who tries it out could send me feedback about whether it seems to do the trick. For now, you can get it at: http://unto.net/archives/IPC-Cache-0.01.tar.gz Apologies if that site is a little slow -- I'll move it over to our real servers and to CPAN in the near future. Thanks! -DeWitt
Re: numerous segv's, bus error
[apologies if people get this twice -- it seemed to bounce the first time] We've been getting many segv's and bus errors on apache processes since adding modperl. Env: Apache 1.3.11 (though we had the same thing under 1.3.9.), Perl 5.005003, modperl 1.21, hpux 11.00. If you compare this to my earlier post this week, I believe we are having the same problem. I've tried rolling back to 1.3.9 as well and had the same problem. I'm running Linux 2.3.39 and 2.2.15, btw -- it's interesting to note that you are having this problem on another system. I've tried every possible configuration of static/dynamic I can think of, too. Setting PerlFreshRestart to "off" didn't seem to do the trick for me either. I tried using the latest cvs snapshot of mod_perl with no luck. I agree -- this is a problem with the cleanup. In a way I'm glad it's not just me, although I'm really sorry you are being bit by this too! Hopefully someone will have an idea. -DeWitt
Re: squid performance
On Fri, 14 Jan 2000 [EMAIL PROTECTED] wrote: Hi, I am switching my modperl site to squid in httpd acclerator mode and everything works as advertised, but was very surprised to find squid 10x slower than apache on a cached 6k gif as measured by apache bench... 100 requests/second vs almost 1000 for apache. I've experienced the same results with accelerating small static files. However, we still use squid on our site (www.eziba.com) because our httpd processes are so heavy, and to save a trip to the database for documents that are cachable, such as product images. We've found that the net performance gain to be noticeable. However, if all you are serving is static files that live on the same tier as the apache server, you'll probably be better of without squid. On that topic, is there an alternative to squid? We are using it exclusively as an accelerator, and don't need 90% of it's admittedly impressive functionality. Is there anything designed exclusively for this purpose? Perhaps a modperl module that implements caching of files based on Expires headers? -DeWitt
Re: squid performance
On Sun, 16 Jan 2000, Steven Lembark wrote: given you have the core to support it... try using libmm and a tied hash to just stash the stuff until it's asked for. Actually, we are currently developing a generic object caching interface that will support a TTL based on expiration dates, and implementing this in both Java and perl. Since the perl version will be used most in a multi-process environment (like apache/modperl) we'll back-end this on mmap. The java version will be used in both our middle tier (a single-process threaded server) and in a servlet implementation of our front-end, which also runs in a single process. Right now, I do cache data in each httpd process, but since I haven't implemented a TTL, I rely on the apache process dying naturally to clean up the data. Obviously, this creates *huge* httpd processes (50+ MB) with non-shared but redundant data, and isn't particularly predictable. Our site is a little different that most ecommerce sites, because we have a very low depth of inventory, and have to keep all the state in the database, so that customers will rarely see out-of-date product data. Ultimately, I hope to open up our entire source tree under the GPL anyway, but I'll try to get the generic caching code out there earlier, if possible. I'll keep this list posted. Unless someone has already done this, that is. -DeWitt
RE: Potential bug in HTML::Embperl 1.2.0
On Thu, 9 Dec 1999, Gerald Richter wrote: The problem is, that Embperl must know, that you want to run in a Safe compartement. Just add 'options' = HTML::Embperl::optSafeNamespace to the Excute parameters and it should work. Right. That makes sense, and it is definitely a good idea to run it with that option, but unfortunately it doesn't address the issue demonstrated by the example code. If anyone has a second, I'd like to see someone else run the script and see if I'm not going crazy and imagining things. Thanks, -DeWitt
RE: Potential bug in HTML::Embperl 1.2.0
On Thu, 9 Dec 1999, Gerald Richter wrote:
Rereading your first mail, seems to me that there is an additional problem
with Embperl's cleanup. This has nothing todo with Safe namespaces. Embperl
will per default tidyup your namespace and therefor execute something like
$vars = undef at the end of the first request.
Try adding the additional parameter
'cleanup' = -1
which will disable cleanup at all, add $CLEANUP{'vars'} = 0 inside your
template, which should disable the cleanup only for that variable.
That will work for this trivial example, but we have 107 templates on our
site, each with 10 to 100 variables, and changing them all would be
difficult, to say the least. Worse, if we missed one, we'd have those
subtle bugs that are only occasionally apparent, but can cause real
damage. Imagine if your credit card number was inadvertently passed to
another customer in the template because we forgot to undef a variable?
I think the right approach is to probably track down the cause of the
variables not being undefined in the first place. I suppose I'll start
hunting down the reason why this is occurring. Wish me luck. Any
pointers as to where to focus are appreciated, of course. :)
Thanks,
-DeWitt
RE: Potential bug in HTML::Embperl 1.2.0
On Thu, 9 Dec 1999, Gerald Richter wrote:
If I had understand you right, the first call via execute to your template
works and the second not, right?
That's not a bug! Embperl cleans up all your global variables inside your
page after every request. $vars in your template is such a global variable,
so Embperl undef it and therfore the content gets lost and it won't work the
second time.
Oddly, that's not quite what is happening. The content isn't lost
altogether, else the simple join() I include wouldn't work either.
Moreover, even if I re-initialize the data passed into my printTemplate()
routine, the second call is still using some of the old data.
Note that this issue is only appearing on array data, and only in the
foreach command.
When I call the template containing a [$ foreach $] for the second time,
every value in that array *in the foreach loop* is returned as the last
value of the array, as called the first time around.
Here's a little test. Try putting this in a template, and Executing on it
twice by the same process. No input data, no parameters, just the
template on it's own.
[- @array = (1, 2, 3) -]
[+ join(",", @array) +]
[$ foreach $arrayElement (@array) $]
[+ $arrayElement +]
[$ endforeach $]
That's it. The first time it will print (I stripped out the newlines):
1,2,3
1
2
3
The second time it will print:
1,2,3
3
3
3
Apparently, this has nothing to do with Safe() or anything else. Just
that the value given to $arrayElement in the [$ foreach $] doesn't seem to
be correct on subsequent executions, that's all.
Sorry to be sticking with this one, but I want to hear from just one
person that I'm not losing my mind. I really do appreciate the help,
though. :)
-DeWitt
RE: Potential bug in HTML::Embperl 1.2.0
On Thu, 9 Dec 1999, Gerald Richter wrote: So the solution to your problem should be, to use a fixed package name for all calls to the _same_ page. Brilliant! This does address exactly the problem I raised, and makes perfect sense. I had to do one absolutely wacky thing, though. Because I call Execute from within templates (indirectly, actually, through our own Template module that wraps Embperl), I needed to extend your idea a little. Essentially, I needed to use a unique Safe namespace for each request, but somehow have it carry over between the included Execute calls. So, I share the Safe compartment's root across calls to Execute. But simply doing that means that when the same file is included and executed by different templates then the package names will mismatch and data won't be shared across included files correctly. I solved this by creating a virtual filename that is unique on a per file *and* a per "root" namespace, but is common across requests, to allow for caching. Does this make any sense to anyone? Probably not. :) I'll go away now, but thanks everyone, especially Gerald, for the help! -DeWitt
