I just had to restart Apache which crashed around 22:22 UTC. There is 8Mb of the above error in the log.
This is the plain Apache2 error.log rather than the named website error log which only shows expected errors such as reCaptcha errors.
It seems to have started filling the scoreboard 24 minutes after logrotate. The first ten log entries after logrotate are ...
[Tue Nov 22 00:00:01.934987 2022] [mpm_event:notice] [pid 49892:tid 140450581641088] AH00489: Apache/2.4.52 (Ubuntu) OpenSSL/3.0.2 mod_wsgi/4.9.0 Python/3.10 configured -- resuming normal operations [Tue Nov 22 00:00:01.935032 2022] [core:notice] [pid 49892:tid 140450581641088] AH00094: Command line: '/usr/sbin/apache2' [Tue Nov 22 00:20:13.411015 2022] [mpm_event:notice] [pid 49892:tid 140450581641088] AH00493: SIGUSR1 received. Doing graceful restart [Tue Nov 22 00:20:13.514771 2022] [mpm_event:notice] [pid 49892:tid 140450581641088] AH00489: Apache/2.4.52 (Ubuntu) OpenSSL/3.0.2 mod_wsgi/4.9.0 Python/3.10 configured -- resuming normal operations [Tue Nov 22 00:20:13.514804 2022] [core:notice] [pid 49892:tid 140450581641088] AH00094: Command line: '/usr/sbin/apache2' [Tue Nov 22 00:23:57.862865 2022] [mpm_event:notice] [pid 49892:tid 140450581641088] AH00493: SIGUSR1 received. Doing graceful restart [Tue Nov 22 00:23:57.937161 2022] [mpm_event:notice] [pid 49892:tid 140450581641088] AH00489: Apache/2.4.52 (Ubuntu) OpenSSL/3.0.2 mod_wsgi/4.9.0 Python/3.10 configured -- resuming normal operations [Tue Nov 22 00:23:57.937192 2022] [core:notice] [pid 49892:tid 140450581641088] AH00094: Command line: '/usr/sbin/apache2' [Tue Nov 22 00:23:59.938800 2022] [mpm_event:error] [pid 49892:tid 140450581641088] AH03490: scoreboard is full, not at MaxRequestWorkers.Increase ServerLimit. [Tue Nov 22 00:24:00.939954 2022] [mpm_event:error] [pid 49892:tid 140450581641088] AH03490: scoreboard is full, not at MaxRequestWorkers.Increase ServerLimit.
Then later ...[Tue Nov 22 22:28:52.526681 2022] [mpm_event:error] [pid 111175:tid 140011041032064] AH03490: scoreboard is full, not at MaxRequestWorkers.Increase ServerLimit. [Tue Nov 22 22:28:52.526899 2022] [mpm_event:notice] [pid 111175:tid 140011041032064] AH00492: caught SIGWINCH, shutting down gracefully [Tue Nov 22 22:28:55.888384 2022] [core:warn] [pid 111175:tid 140011041032064] AH00045: child process 113420 still did not exit, sending a SIGTERM
... more of the same ...[Tue Nov 22 22:28:59.893773 2022] [core:warn] [pid 111175:tid 140011041032064] AH00045: child process 113426 still did not exit, sending a SIGTERM [Tue Nov 22 22:29:01.896130 2022] [core:error] [pid 111175:tid 140011041032064] AH00046: child process 113420 still did not exit, sending a SIGKILL
... more of the same ...[Tue Nov 22 22:29:01.896389 2022] [core:error] [pid 111175:tid 140011041032064] AH00046: child process 113426 still did not exit, sending a SIGKILL
... I assume this is where I said restart ...[Tue Nov 22 22:29:03.055559 2022] [mpm_event:notice] [pid 122709:tid 139966049515392] AH00489: Apache/2.4.52 (Ubuntu) OpenSSL/3.0.2 mod_wsgi/4.9.0 Python/3.10 configured -- resuming normal operations [Tue Nov 22 22:29:03.055753 2022] [core:notice] [pid 122709:tid 139966049515392] AH00094: Command line: '/usr/sbin/apache2'
... and no errors since.Below is the server-status after restart and below that is my current mpm_event.conf
Many thanks MikeServer Version: Apache/2.4.52 (Ubuntu) OpenSSL/3.0.2 mod_wsgi/4.9.0 Python/3.10
Server MPM: event Server Built: 2022-09-30T04:09:50 ------------------------------------------------------------------------ Current Time: Tuesday, 22-Nov-2022 22:41:35 UTC Restart Time: Tuesday, 22-Nov-2022 22:29:03 UTC Parent Server Config. Generation: 1 Parent Server MPM Generation: 0 Server uptime: 12 minutes 32 seconds Server load: 0.00 0.00 0.00 Total accesses: 14 - Total Traffic: 121 kB - Total Duration: 10274 CPU Usage: u10.91 s.38 cu0 cs0 - 1.5% CPU load .0186 requests/sec - 164 B/second - 8.6 kB/request - 733.857 ms/request 1 requests currently being processed, 449 idle workers Slot PID Stopping Connections Threads Async connections total accepting busy idle writing keep-alive closing 0 122713 no 0 yes 0 50 0 0 0 1 122714 no 0 yes 0 50 0 0 0 2 122715 no 0 yes 0 50 0 0 0 3 122716 no 0 yes 0 50 0 0 0 4 122717 no 0 yes 0 50 0 0 0 5 123029 no 0 yes 0 50 0 0 0 6 123082 no 0 yes 0 50 0 0 0 7 123083 no 0 yes 1 49 0 0 0 8 123190 no 0 yes 0 50 0 0 0 Sum 9 0 0 1 449 0 0 0 # event MPM # ServerLimit: Upper limit on configurable number of processes (default 16) # StartServers: initial number of server processes to start (default 3)# MinSpareThreads: minimum number of worker threads which are kept spare (default 75) # MaxSpareThreads: maximum number of worker threads which are kept spare (default 250) # ThreadLimit: upper limit on the configurable number of threads per child process (default 64) # ThreadsPerChild: constant number of worker threads in each server process (default 25) # MaxRequestWorkers: maximum number of worker threads (ServerLimit*ThreadsPerChild)
# MaxConnectionsPerChild: maximum number of requests a server process serves ### After reading https://www.liquidweb.com/kb/apache-performance-tuning-apache-mpm-modules/
<IfModule mpm_event_module> ServerLimit 16 StartServers 16 MinSpareThreads 400 MaxSpareThreads 250 ThreadLimit 64 ThreadsPerChild 50 AsyncRequestWorkerFactor 2 MaxRequestWorkers 800 MaxConnectionsPerChild 0 </IfModule> -- Signed email is an absolute defence against phishing. This email has been signed with my private key. If you import my public key you can automatically decrypt my signature and be sure it came from me. Just ask and I'll send it to you. Your email software can handle signing.
OpenPGP_signature
Description: OpenPGP digital signature