Re: Dollar Self Storage (aka mod_perl children sharing/updating data on a schedule)

2011-01-26 Thread Jeff Anderson
Greetings,

First, a big thank you to everyone for these great suggestions and
corrections. The idea to send a HUP signal to the parent process works fine
under the pre-forked server model, but does not work at until under the
threaded worker model. So signals are right out.

I probably forgot to mention that we are also dealing with multiple servers,
so a central data store will be required.

We finally decided to go with a convention -- send as many "internal"
specialized requests as there are child proceses to each server seems to
working well enough. We are not interested in precious, just pretty darned
good accuracy.

Please feel free to keep this discussion open, ask questions or make further
suggestions.

Thank you all once again!
jeffa



On Thu, Jan 13, 2011 at 7:12 AM, Perrin Harkins  wrote:

> Hi Jeff,
>
> > I am looking to set up a mod_perl handler which keep track of the
> > count of requests coming in. Each child process will store this data
> > in local memory and after 5-10 minutes have passed, each child process
> > will merge its data into a central database, the goal being that each
> > child will not have to hit a database for every request.
>
> I agree with the people saying that memcached/Cache::FastMmap or an
> in-memory file is probably fast enough to hit on every request.  In
> general though, storing and dumping things to a db now and then is not
> a bad way to go for non-critical data.
>
> > The problem is --- how do i additionally have each child merge its
> > data on a schedule -- that is, without relying only on an incoming
> > request to "hit" that specific child process?
>
> You can't.  The nature of apache is that it responds to network
> events.  Cleanup handlers are also only going to fire after a request.
>  You could rig up a cron to hit your server regularly and if the data
> was shared between the children then whatever child picked that up
> could write it to the db, but that seems a lot harder than
> alternatives already suggested.
>
> > Attempt #2 --- register a Clean Up hook. This doesn't seem to work for
> > me because, as i understand so far, assigning a reference to a sub via
> > PerlCleanupHandler is not the same as calling the object's method.
>
> You could just store this data in a $My::Global::Counter and read it
> from anywhere.  Each child has its own variable storage, so this is
> safe.
>
> Second, you should be able to make a cleanup handler call your sub as a
> method
>
> > - Perhaps each child process will need to use it's own SQLite or similar
> cache?
>
> SQLite may well be slower than your real database, so I wouldn't do
> that without testing.
>
> BTW, how are you configuring a handler to create a $self that lasts
> across multiple requests?
>
> - Perrin
>



-- 
jeffa


experiencing Out of memory errors

2011-01-26 Thread John Deighan

We have an Apache/mod_perl application running under Windows Server 2003
that periodically experiences
"Out of memory" errors (they appear in the Apache error logs) which
forces Apache to restart. Our application does, in fact, use a lot of
memory, and we believe that this is not due to a bug or memory leak.
What we would like is to run this application in a 64 bit environment, thus
allowing us to use more than the 2 GB memory that the Apache process ('httpd')
is limited to (the "Out of memory" errors always occur as the memory usage
of the httpd process approaches this 2 GB limit).

We know that we will need to run a 64 bit version of Windows on 64 bit
hardware to accomplish this. Furthermore, we know that ActiveState has a
binary Perl install of a 64 bit Perl. However, we're not aware of either a
64 bit Apache (though recently I found the blackdot site -
http://www.blackdot.be/ - that provides a Windows compatible binary build
of 64 bit apache, though I know very little about it) or a 64 bit mod_perl.
However, we're also not sure if either of these are needed to escape the
2 GB memory limitation. E.g., can 64 bit Perl be used with a 32 bit
Apache and/or mod_perl?

Any help with accomplishing what we need would be greatly appreciated,
including the possibility of hiring someone on a contract basis to
help us.



OS: Microsoft Windows Server 2003
Perl: v 5.8.9 ActiveState build 826
Apache 2.2.11
mod_perl 2 (not sure of the exact version, but very recent)