On Mon, Mar 15, 2010 at 11:26 PM, Pavel Georgiev <pa...@3tera.com> wrote: > I have a perl script running in mod_perl that needs to write a large amount > of data to the client, possibly over a long period. The behavior that I > observe is that once I print and flush something, the buffer memory is not > reclaimed even though I rflush (I know this cant be reclaimed back by the OS). > > Is that how mod_perl operates and is there a way that I can force it to > periodically free the buffer memory, so that I can use that for new buffers > instead of taking more from the OS?
That is how Perl operates. Mod_Perl is just Perl embedded in the Apache Process. You have a few options: * Buy more memory. :) * Delegate resource intensive work to a different process (I would NOT suggest a forking a child in Apache). * Tie the buffer to a file on disk, or db object, that can be explicitly reclaimed * Create a buffer object of a fixed size and loop. * Use compression on the data stream that you read into a buffer. You could also architect your system to mitigate resource usage if the large data serve is not a common operation: * Proxy those requests to a different server which is optimized to handle large data serves. * Execute the large data serves with CGI rather than Mod_Perl. I'm sure there are probably other options as well. -wjt
