Re: Cache refresh each 50 queries?

2016-10-05 Thread SUZUKI Arthur

Hello David, André,

the server we're working with is fully dedicated to Koha.

Thank you André for your hints about Apache configurations.

@David, same thing, I'll try to preload C4::ILSDI:Services and see if it 
helps.


Thanks to both of you, I'll let you know about the results.
Cheers,
Arthur

Le 05/10/2016 à 01:20, David Cook a écrit :

Sounds like André is on the right track. I've certainly run into a similar
issue (with a non-Koha app).

However, because I was using the Catalyst framework, I was able to just
preload the entire app, so that the Perl modules were loaded into the Apache
master process before forking, and that did the trick. That app is a lot
smaller than Koha though too. This case is a bit more complicated since Koha
isn't really a MVC app, but you could look at the Koha Plack examples and
see which modules they pre-load.

You might also want to run ilsdi.pl with Devel::NYTProf to identify what
exactly is slowing down that 1s long query.

I've only played with PerlOptions +Parent a bit, but I'm guessing that you
have multiple mod_perl apps, and that we're not actually seeing your entire
httpd.conf configuration relating to Koha, right?

David Cook
Systems Librarian
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia

Office: 02 9212 0899
Direct: 02 8005 0595



-Original Message-
From: André Warnier [mailto:a...@ice-sa.com]
Sent: Tuesday, 4 October 2016 10:02 PM
To: modperl@perl.apache.org
Subject: Re: Cache refresh each 50 queries?

Hi.

On 04.10.2016 10:17, SUZUKI Arthur wrote:
[...]


On 03.10.2016 16:57, SUZUKI Arthur wrote:

Hello List,
I'm sending this message to know if there are some hints/tips to
help with the problem we're facing.
The problem is that for a same query repeated over time, reply time
can be as short as 5ms and as long as 1s.

Since there is neither correlation with CPU load or RAM usage, nor
any networking constraints, we think it is due to the cache refresh
mechanism.

The problem occurs at random times, with a probability of around
1/50 queries (empirical data).

Is there any configuration option which could help?

Our current httpd.conf contains the following:

PerlModule ModPerl::Registry
PerlOptions +Parent
PerlSwitches -I/home/koha/src


SetHandler perl-script
# more faster, link with worker
PerlResponseHandler ModPerl::Registry
# less faster, link with prefork
#PerlResponseHandler ModPerl::PerlRun
Options +ExecCGI
PerlOptions +ParseHeaders


Thanks a lot in advance for your replies.
Best regards,

You still do not provide enough information about your server

configuration

(the part not mod_perl specific) to really make a narrow guess, but let me

try

anyway :

One thing that might explain a seemingly-long processing time for a

request

approximately once in 50 calls, is if there was something special and
resource-intensive which happens once per approximately 50 calls, right ?

One thing which /might/ explain this, /perhaps/ and depending on your
configuration, is if approximately once in 50 calls, Apache had to fork a

new

child process, to handle the call. This new child process would then start

with

a brand-new, fresh Perl interpreter, which might need to recompile some
modules before it even starts handling the request.

Look for example here :
http://perl.apache.org/docs/2.0/user/config/config.html#C_Parent_
and your configuration line above :
   PerlOptions +Parent

In that configuration, the cgi-bin script which you are running (and all

its

dependencies), is not compiled by the embedded perl until it has been run

at

least once.
So the first execution will take more time than the following ones.
Now if the rest of your configuration is so that every 50 requests or so,
Apache starts a new child with a new perl interpreter, that may explain

the

symptoms that you are observing.

When Apache handles a request (if it is a "prefork" MPM), it will look for

a

child Apache which is free, to pass the request for execution.  If the

server is

lightly loaded, it may be that the child to which it passes the next

request is

always the same, because it is always done with the previous request, and
free, when the next request comes in.
But if this child process somehow has a limit to how many requests it may
process before it dies, and this limit was around 50, then somewhere

around

the 50th request, another child would get the next request, and for this

child

it would be the first one (or the first one with /this/ cgi-bin script).

There are roughly similar phenomenons that might happen in other kinds of
Apache MPM's, which is why it is important to know which one you are

using,

and with which setup parameters.

And yes, there are ways to improve this.  But again, we would need to know
more about your configuration in order to make suggestions.





--
Arthur SUZUKI
Service informatique des bibliothèques
BIBLIOTHÈQUES UNIVERS

Cache refresh each 50 queries?

2016-10-04 Thread SUZUKI Arthur

Re-posted with a suitable title (thank you André Warnier)

Regards,

Arthur Suzuki

On 03.10.2016 16:57, SUZUKI Arthur wrote:

Hello List,
I'm sending this message to know if there are some hints/tips to help 
with the problem we're facing.

The problem is that for a same query repeated over time,
reply time can be as short as 5ms and as long as 1s.

Since there is neither correlation with CPU load or RAM usage, nor 
any networking constraints,

we think it is due to the cache refresh mechanism.

The problem occurs at random times, with a probability of around 1/50 
queries (empirical

data).

Is there any configuration option which could help?

Our current httpd.conf contains the following:

PerlModule ModPerl::Registry
PerlOptions +Parent
PerlSwitches -I/home/koha/src


   SetHandler perl-script
   # more faster, link with worker
   PerlResponseHandler ModPerl::Registry
   # less faster, link with prefork
   #PerlResponseHandler ModPerl::PerlRun
   Options +ExecCGI
   PerlOptions +ParseHeaders


Thanks a lot in advance for your replies.
Best regards,


--
Arthur SUZUKI
Service informatique des bibliothèques
BIBLIOTHÈQUES UNIVERSITAIRES
Université Jean Moulin Lyon 3
6 Cours Albert Thomas - B.P. 8242 – 69355 Lyon Cedex 08
ligne directe : +33 (0)4 78 78 79 16 | http://bu.univ-lyon3.fr
L'Université Jean Moulin est membre fondateur de l'Université de Lyon



Re: WELCOME to modperl@perl.apache.org

2016-10-03 Thread SUZUKI Arthur

Hello List,
I'm sending this message to know if there are some hints/tips to help 
with the problem we're facing.

The problem is that for a same query repeated over time,
reply time can be as short as 5ms and as long as 1s.

Since there is neither correlation with CPU load or RAM usage, nor with 
networking constraints,

we think it is due to the cache refresh mechanism.

The problem occurs at random times, with a probability of around 1/50 
queries (empirical data).


Is there any configuration option which could help?

Our current httpd.conf contains the following:

PerlModule ModPerl::Registry
PerlOptions +Parent
PerlSwitches -I/home/koha/src


  SetHandler perl-script
  # more faster, link with worker
  PerlResponseHandler ModPerl::Registry
  # less faster, link with prefork
  #PerlResponseHandler ModPerl::PerlRun
  Options +ExecCGI
  PerlOptions +ParseHeaders


Thanks a lot in advance for your replies.
Best regards,

--
Arthur SUZUKI
Service informatique des bibliothèques
BIBLIOTHÈQUES UNIVERSITAIRES
Université Jean Moulin Lyon 3
6 Cours Albert Thomas - B.P. 8242 – 69355 Lyon Cedex 08
ligne directe : +33 (0)4 78 78 79 16 | http://bu.univ-lyon3.fr
L'Université Jean Moulin est membre fondateur de l'Université de Lyon