There hasn't been enough usage on the network to cause my crash to happen
again yet, but I thought maybe this could help diagnose the problem. Here
is a snippit of httpd.webservices.error around the timeframe of the crashes.
[Sun Sep 28 14:01:16 2014] [notice] Apache/2.2.15 (Unix) mod_ssl/2.2.15
OpenSSL/1.0.1e-fips mod_perl/2.0.4 Perl/v5.10.1 configured -- resuming
normal operations
[Sun Sep 28 14:02:21 2014] [error] server reached MaxClients setting,
consider raising the MaxClients setting
[Sun Sep 28 14:06:24 2014] [emerg] (4)Interrupted system call: couldn't
grab the accept mutex
[Sun Sep 28 14:06:24 2014] [alert] Child 11638 returned a Fatal error...
Apache is exiting!
[Sun Sep 28 14:06:46 2014] [emerg] (4)Interrupted system call: couldn't
grab the accept mutex
[Sun Sep 28 14:06:47 2014] [emerg] (4)Interrupted system call: couldn't
grab the accept mutex
[Sun Sep 28 14:17:24 2014] [emerg] (4)Interrupted system call: couldn't
grab the accept mutex
[Sun Sep 28 14:17:25 2014] [emerg] (4)Interrupted system call: couldn't
grab the accept mutex
Seeing the "Server reached MaxClients setting" error as well as the crazy
amount of connections that are happening per minute in the
httpd.webservices.access file makes me wonder if this is the problem.
Here is a snippit of that log around the same time frame:
127.0.0.1 - - [28/Sep/2014:14:03:17 -0500] "OPTIONS * HTTP/1.0" 200 - "-"
"Apache (internal dummy connection)"
127.0.0.1 - radius_accounting [28/Sep/2014:14:03:15 -0500] "POST /
HTTP/1.1" 200 35 "-" "-"
127.0.0.1 - radius_accounting [28/Sep/2014:14:03:15 -0500] "POST /
HTTP/1.1" 200 35 "-" "-"
127.0.0.1 - radius_authorize [28/Sep/2014:14:03:03 -0500] "POST / HTTP/1.1"
200 66 "-" "-"
127.0.0.1 - radius_authorize [28/Sep/2014:14:02:03 -0500] "POST / HTTP/1.1"
200 66 "-" "-"
127.0.0.1 - radius_authorize [28/Sep/2014:14:02:19 -0500] "POST / HTTP/1.1"
200 66 "-" "-"
127.0.0.1 - radius_accounting [28/Sep/2014:14:03:15 -0500] "POST /
HTTP/1.1" 200 35 "-" "-"
127.0.0.1 - radius_accounting [28/Sep/2014:14:03:15 -0500] "POST /
HTTP/1.1" 200 35 "-" "-"
127.0.0.1 - radius_authorize [28/Sep/2014:14:01:37 -0500] "POST / HTTP/1.1"
200 66 "-" "-"
127.0.0.1 - radius_authorize [28/Sep/2014:14:03:06 -0500] "POST / HTTP/1.1"
200 66 "-" "-"
127.0.0.1 - radius_authorize [28/Sep/2014:14:03:04 -0500] "POST / HTTP/1.1"
200 66 "-" "-"
127.0.0.1 - radius_authorize [28/Sep/2014:14:03:04 -0500] "POST / HTTP/1.1"
200 66 "-" "-"
127.0.0.1 - update_iplog [28/Sep/2014:14:03:17 -0500] "POST / HTTP/1.1" 204
- "-" "-"
127.0.0.1 - trigger_violation [28/Sep/2014:14:03:17 -0500] "POST /
HTTP/1.1" 204 - "-" "-"
127.0.0.1 - trigger_violation [28/Sep/2014:14:03:17 -0500] "POST /
HTTP/1.1" 204 - "-" "-"
127.0.0.1 - radius_authorize [28/Sep/2014:14:01:37 -0500] "POST / HTTP/1.1"
200 66 "-" "-"
127.0.0.1 - - [28/Sep/2014:14:03:18 -0500] "OPTIONS * HTTP/1.0" 200 - "-"
"Apache (internal dummy connection)"
127.0.0.1 - - [28/Sep/2014:14:03:19 -0500] "OPTIONS * HTTP/1.0" 200 - "-"
"Apache (internal dummy connection)"
127.0.0.1 - radius_authorize [28/Sep/2014:14:03:04 -0500] "POST / HTTP/1.1"
200 66 "-" "-"
127.0.0.1 - update_iplog [28/Sep/2014:14:03:19 -0500] "POST / HTTP/1.1" 204
- "-" "-"
127.0.0.1 - update_iplog [28/Sep/2014:14:03:19 -0500] "POST / HTTP/1.1" 204
- "-" "-"
127.0.0.1 - radius_authorize [28/Sep/2014:14:03:17 -0500] "POST / HTTP/1.1"
200 66 "-" "-"
127.0.0.1 - radius_authorize [28/Sep/2014:14:03:17 -0500] "POST / HTTP/1.1"
200 66 "-" "-"
127.0.0.1 - - [28/Sep/2014:14:03:20 -0500] "OPTIONS * HTTP/1.0" 200 - "-"
"Apache (internal dummy connection)"
127.0.0.1 - - [28/Sep/2014:14:03:21 -0500] "OPTIONS * HTTP/1.0" 200 - "-"
"Apache (internal dummy connection)"
127.0.0.1 - - [28/Sep/2014:14:03:22 -0500] "OPTIONS * HTTP/1.0" 200 - "-"
"Apache (internal dummy connection)"
127.0.0.1 - trigger_violation [28/Sep/2014:14:03:22 -0500] "POST /
HTTP/1.1" 204 - "-" "-"
127.0.0.1 - trigger_violation [28/Sep/2014:14:03:22 -0500] "POST /
HTTP/1.1" 204 - "-" "-"
127.0.0.1 - - [28/Sep/2014:14:03:23 -0500] "OPTIONS * HTTP/1.0" 200 - "-"
"Apache (internal dummy connection)"
127.0.0.1 - - [28/Sep/2014:14:03:24 -0500] "OPTIONS * HTTP/1.0" 200 - "-"
"Apache (internal dummy connection)"
127.0.0.1 - update_iplog [28/Sep/2014:14:03:24 -0500] "POST / HTTP/1.1" 204
- "-" "-"
127.0.0.1 - - [28/Sep/2014:14:03:25 -0500] "OPTIONS * HTTP/1.0" 200 - "-"
"Apache (internal dummy connection)"
127.0.0.1 - - [28/Sep/2014:14:03:26 -0500] "OPTIONS * HTTP/1.0" 200 - "-"
"Apache (internal dummy connection)"
127.0.0.1 - - [28/Sep/2014:14:03:27 -0500] "OPTIONS * HTTP/1.0" 200 - "-"
"Apache (internal dummy connection)"
127.0.0.1 - - [28/Sep/2014:14:03:28 -0500] "OPTIONS * HTTP/1.0" 200 - "-"
"Apache (internal dummy connection)"
127.0.0.1 - - [28/Sep/2014:14:03:29 -0500] "OPTIONS * HTTP/1.0" 200 - "-"
"Apache (internal dummy connection)"
On Mon, Sep 29, 2014 at 6:09 PM, Brian Lucas <[email protected]> wrote:
> Will do and post back. Right now I have all services restarting one an
> hour from cron to avoid disruption to my users. Will have to wait until an
> odd hour.
>
> Brian
> On Sep 29, 2014 3:22 PM, "Louis Munro" <[email protected]> wrote:
>
>>
>>
>> On 2014-09-26, at 19:19 , Brian Lucas <[email protected]> wrote:
>>
>> All,
>>
>> I'm seeing 1000s of the following error per hour on our setup after the
>> update to 4.4. A restart of all services clears it up for a time, but it
>> comes back. Any suggestions as to the problem?
>>
>> Fri Sep 26 18:16:45 2014 : Error: rlm_perl: An error occurred while
>> processing the authorize RPC request: An error occured while sending a
>> MessagePack request: 7 Couldn't connect to server couldn't connect to host
>> at /usr/local/pf/lib//pf/radius/rpc.pm line 52.
>>
>>
>> Hi Brian,
>>
>> Next time this happens, try to see why radius could not connect to the
>> webservice.
>>
>> By default, the webservice runs on port 9090 on localhost.
>> If you try to connect to it yourself, does it work?
>>
>> e.g. run this command:
>>
>> # curl -kvLI http://localhost:9090
>>
>> And see what it says.
>>
>> --
>> Louis Munro
>> [email protected] :: www.inverse.ca
>> +1.514.447.4918 x125 :: +1 (866) 353-6153 x125
>> Inverse inc. :: Leaders behind SOGo (www.sogo.nu) and PacketFence (
>> www.packetfence.org)
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Slashdot TV. Videos for Nerds. Stuff that Matters.
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=160591471&iu=/4140/ostg.clktrk
>> _______________________________________________
>> PacketFence-users mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/packetfence-users
>>
>>
------------------------------------------------------------------------------
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk
_______________________________________________
PacketFence-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/packetfence-users
