This can be a bigger gain if you are limited with memory as multiple processes
will share the same block of physical memory { so limiting swap } – this is one
of the advantages of the mod_perl approach over the app approach in things like
dancer. You have the flexibility to include what you want – if you have a
single webpage or service which needs a large number of modules – but is
perhaps used once in every 100k requests you don’t have to load it in at the
start of the process, and you have the flexibility to choose how much or how
little you want of the framework preloaded…
From: Adam Prime <adam.pr...@utoronto.ca>
Sent: 07 February 2021 13:45
To: Steven Haigh <net...@crc.id.au>
Cc: James Smith <j...@sanger.ac.uk>; modperl@perl.apache.org
Subject: Re: [EXT]
There is one other thing you can do relatively easily that may get you a
marginal gain when Apache spins up new children. Load some or all of your Perl
dependencies before Apache forks. There is also an opportunity to load other
static resources at this time, but that can get a little more involved and/or
confusing. There is some documentation here:
https://perl.apache.org/docs/2.0/user/handlers/server.html#Startup_File
[perl.apache.org]<https://urldefense.proofpoint.com/v2/url?u=https-3A__perl.apache.org_docs_2.0_user_handlers_server.html-23Startup-5FFile&d=DwMFaQ&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=oH2yp0ge1ecj4oDX0XM7vQ&m=kwRJ9OU2GaHG3eTqlSv3NGFCM8bbDVVw6VWYGivfszw&s=sT5APUQZ6MWHqhwLVLrTEK4xPf6OEANKV0EePFJ1L6Y&e=>
Adam
On Feb 7, 2021, at 8:15 AM, Steven Haigh
<net...@crc.id.au<mailto:net...@crc.id.au>> wrote:
In fact, I just realised that 'ab' test is rather restrictive.... So here's a
bit more of an extended test:
# ab -k -n 1000 -c 32
Apache + ExecCGI:
Requests per second: 14.26 [#/sec] (mean)
Time per request: 2244.181 [ms] (mean)
Time per request: 70.131 [ms] (mean, across all concurrent requests)
Apache + mod_perl (ModPerl::PerlRegistry):
Requests per second: 132.14 [#/sec] (mean)
Time per request: 242.175 [ms] (mean)
Time per request: 7.568 [ms] (mean, across all concurrent requests)
Interestingly, without Keepalives, the story is much the same:
# ab -n 1000 -c 32
Apache + ExecCGI:
Requests per second: 14.15 [#/sec] (mean)
Time per request: 2260.875 [ms] (mean)
Time per request: 70.652 [ms] (mean, across all concurrent requests)
Apache + mod_perl (ModPerl::PerlRegistry):
Requests per second: 154.48 [#/sec] (mean)
Time per request: 207.140 [ms] (mean)
Time per request: 6.473 [ms] (mean, across all concurrent requests)
Running some benchmarks across various parts of my site made me realise I also
had some RewriteRules in the apache config that still had H=cgi-script -
changed those to H=perl-script and saw similar improvements:
ExecCGI - Requests per second: 11.84 [#/sec] (mean)
mod_perl - Requests per second: 130.97 [#/sec] (mean)
That's quite some gains for a days work.
--
Steven Haigh 📧 net...@crc.id.au<mailto:net...@crc.id.au> 💻
https://www.crc.id.au
[crc.id.au]<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.crc.id.au_&d=DwMFaQ&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=oH2yp0ge1ecj4oDX0XM7vQ&m=kwRJ9OU2GaHG3eTqlSv3NGFCM8bbDVVw6VWYGivfszw&s=xsyidls-ez8g3gIshJPO4FIGzGjiIp_qnzQ1yujeNK4&e=>
On Sun, Feb 7, 2021 at 23:58, Steven Haigh
<net...@crc.id.au<mailto:net...@crc.id.au>> wrote:
Interestingly, I did get things working with ModPerl::PerlRegistry.
What I couldn't find *anywhere* is that the data I was loading in Template
Toolkit was included in the file in the __DATA__ area - which causes mod_perl
to fall over!
The only way I managed to find this was the following error in the *system*
/var/log/httpd/error_log (didn't show up in the vhost error_log!):
readline() on unopened filehandle DATA at
/usr/lib64/perl5/vendor_perl/Template/Provider.pm line 638.
Took me a LONG time to find a vague post that reading in lines from <DATA>
kills mod_perl. Not sure why - but I stripped all the templates out and put
them in a file instead and re-wrote that bit of code, and things started
working.
I had to fix a few lib path issues, but after getting my head around that, most
things seem to work as before - however I don't notice much of an improvement
in execution times, I do see this improvement using 'ab -n 100 -c32':
Apache + ExecCGI: Requests per second: 13.50 [#/sec] (mean)
Apache + mod_perl: Requests per second: 59.81 [#/sec] (mean)
This is obviously a good thing.
I haven't gotten into the preload or DBI sharing yet - as that'll end up
needing a bit of a rewrite of code to take advantage of. I'd be open to
suggestions here from those who have done it in the past to save me going down
some dead ends :D
--
Steven Haigh 📧 net...@crc.id.au<mailto:net...@crc.id.au> 💻
https://www.crc.id.au
[crc.id.au]<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.crc.id.au_&d=DwMFaQ&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=oH2yp0ge1ecj4oDX0XM7vQ&m=kwRJ9OU2GaHG3eTqlSv3NGFCM8bbDVVw6VWYGivfszw&s=xsyidls-ez8g3gIshJPO4FIGzGjiIp_qnzQ1yujeNK4&e=>
On Sun, Feb 7, 2021 at 12:49, James Smith
<j...@sanger.acuk<mailto:j...@sanger.acuk>> wrote:
As welsey said – try Registry, that was the standard way of using mod_perl to
cache perl in the server – but your problem might be due to the note in
PerlRun…
https://perl.apache.org/docs/2.0/api/ModPerl/PerlRun.html#Description
[perl.apache.org]<https://urldefense.proofpoint.com/v2/url?u=https-3A__perl.apache.org_docs_2.0_api_ModPerl_PerlRun.html-23Description&d=DwMFaQ&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=oH2yp0ge1ecj4oDX0XM7vQ&m=kwRJ9OU2GaHG3eTqlSv3NGFCM8bbDVVw6VWYGivfszw&s=Q2A6j7uCIWTtGhG1FpeG0VpNOpux6GBFmS0kpJslum4&e=>
META: document that for now we don't chdir() into the script's dir, because it
affects the whole process under threads. ModPerl::PerlRunPrefork
[perl.apache.org]<https://urldefense.proofpoint.com/v2/url?u=https-3A__perl.apache.org_docs_20_api_ModPerl_PerlRunPrefork.html&d=DwMFaQ&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=oH2yp0ge1ecj4oDX0XM7vQ&m=kwRJ9OU2GaHG3eTqlSv3NGFCM8bbDVVw6VWYGivfszw&s=G7IOIdWoxLg27VmZ1YKmv0HfFWgABanbrrJUbQ1OvwQ&e=>
should be used by those who run only under prefork MPM.
{tbh most people don’t use mod perl under threads anyway as there isn’t really
a gain from using them}
It suggests you use ModPerl/PerlRunPrefork – as this does an additional step to
cd to the script directory – which might be your issue….
From: Steven Haigh <net...@crc.id.au<mailto:net...@crc.id.au>>
Sent: 07 February 2021 01:00
To: modperl@perl.apache.org<mailto:modperl@perl.apache.org>
Subject: Moving ExecCGI to mod_perl - performance and custom 'modules' [EXT]
Hi all,
So for many years I've been slack and writing perl scripts to do various things
- but never needed more than the normal apache +ExecCGI and Template Toolkit.
One of my sites has become a bit more popular, so I'd like to spend a bit of
time on performance. Currently, I'm seeing ~300-400ms of what I believe to be
execution time of the script loading, running, and then blatting its output to
STDOUT and the browser can go do its thing.
I believe most of the delay would be to do with loading perl, its modules etc
etc
I know that the current trend would be to re-write the entire site in a more
modern, daemon based solution - and I started down the Mojolicious path - but
the amount of re-writing to save 1/3rd of a second seems to be excessive
Would I be correct in thinking that mod_perl would help in this case?
I did try a basic test, but I have a 'use functions' in all my scripts that
loads a .pm with some global vars and a lot of common subs - and for whatever
reason (can't find anything on Google as to why), none of the subs are
recognised in the main script when loaded via ModPerl::PerlRun.
So throwing it out to the list - am I on the right track? wasting my time? or
just a simple mistake?
--
Steven Haigh 📧 net...@crc.id.au<mailto:net...@crc.id.au> 💻
https://www.crc.id.au
[crc.id.au]<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.crc.id.au_&d=DwMFaQ&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=oH2yp0ge1ecj4oDX0XM7vQ&m=bosoTbkecbnrPukObNK-5Duc1p3JTllIM7_FHhBYKW4&s=vQDi0ezyEZDOz86GVraerPdT76UjN2in3UdPh8fglRM&e=>
-- The Wellcome Sanger Institute is operated by Genome Research Limited, a
charity registered in England with number 1021457 and a company registered in
England with number 2742969, whose registered office is 215 Euston Road,
London, NW1 2BE.
--
The Wellcome Sanger Institute is operated by Genome Research
Limited, a charity registered in England with number 1021457 and a
company registered in England with number 2742969, whose registered
office is 215 Euston Road, London, NW1 2BE.
