Hi all,
We're having a problem with our Apache processes
getting stuck inside an apr_poll() call. This is
happening on a production box and it only seems to
make itself a problem when the load gets up to > ~10
requests per second.
We're using Apache 2.0.46 (RedHat patch, prefork
version) in conjunction with PHP 4.3.2 (RedHat
patchlevel 23) on a dual-processor 3.6 GHz machine.
We have 2 gig of memory and a 36Mb RAID array, running
under RedHat Linux AS3, kernel 2.4.21-32.ELsmp.
In general, processes seem to get hung at random
inside the apr_poll() call. I'm no Apache expert, but
on a quick read of the stack trace, it looks like the
system is trying to wait for the socket write to
complete... except that it never does.
We're using PHP sessions, so when one server process
hangs, it leaves an exclusive lock open on the session
file. If the user presses reload or attempts to
access another page, the following HTTPD processes
will block on accessing the session file. This
quickly snowballs and puts us up past our MaxClients
setting, which shuts down the server.
I can't figure out how to reproduce the problem. It's
taken out our server twice so far under heavy system
load, but I'm unable to make it happen at will. Once,
I was able to repro the bug from my own browser. On
the client's side, it submitted a page request for
"/", and it just hung there waiting for a response
(and got nothing).
If I wait during peak hours when lots of people are
using the machine, I can usually see a couple cases of
this from the server, but I've been strictly reactive
so far. The PHP script and/or page being accessed
does not seem to matter.
We recently changed hardware and upgraded from Apache
1.3 to 2.0, so those could be potential factors.
(However, we have two identical web servers that both
exhibit the same problem, so a hardware failure is
unlikely.)
We might have seen a similar problem (with lots of
HTTPD processes hanging around) under Apache 1.3 in
the old environment. Unfortunately, our upgrade to
new hardware also coincides with a heavy increase in
traffic to our sites, so I can't necessarily point my
finger one way or another.
Can anyone make heads or trails out of the following
GDB stack trace? I rebuilt libapr and httpd with
symbols so I could see what was going on, and I
managed to attach a debugger to the session and
capture the following. Even if I let it run for a
while, it stays stuck inside the poll() call:
#0 0x00262689 in poll () from /lib/tls/libc.so.6
#1 0x00983c54 in apr_poll () from
/usr/lib/libapr-0.so.0
#2 0x009843a2 in apr_wait_for_io_or_timeout () from
/usr/lib/libapr-0.so.0
#3 0x00979d73 in apr_socket_sendv () from
/usr/lib/libapr-0.so.0
#4 0x0097a244 in apr_sendv () from
/usr/lib/libapr-0.so.0
#5 0x0807bb65 in writev_it_all (s=0x9a80e20,
vec=0xbfffa990, nvec=2, len=44839, nbytes=0xbfffa8e8)
at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/server/core.c:2864
#6 0x0807d038 in core_output_filter (f=0x9a812b0,
b=0x9cca288)
at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/server/core.c:4170
#7 0x0807483c in ap_pass_brigade (next=0x7531,
bb=0x1)
at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/server/util_filter.c:541
#8 0x08062d11 in ap_http_header_filter (f=0xa046638,
b=0x9a8bfb0)
at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/modules/http/http_protocol.c:1716
#9 0x0807483c in ap_pass_brigade (next=0x7531,
bb=0x1)
at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/server/util_filter.c:541
#10 0x08076edf in ap_content_length_filter
(f=0xa046620, b=0x9a8bfb0)
at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/server/protocol.c:1270
#11 0x0807483c in ap_pass_brigade (next=0x7531,
bb=0x1)
at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/server/util_filter.c:541
#12 0x08064aa4 in ap_byterange_filter (f=0xa046608,
bb=0x9a8bfb0)
at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/modules/http/http_protocol.c:3001
#13 0x0807483c in ap_pass_brigade (next=0x7531,
bb=0x1)
at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/server/util_filter.c:541
#14 0x0114fa2b in execute () from
/etc/httpd/modules/libphp4.so
#15 0x01122195 in php_ob_get_length () from
/etc/httpd/modules/libphp4.so
#16 0x0112222f in php_ob_get_length () from
/etc/httpd/modules/libphp4.so
#17 0x0112143f in php_end_ob_buffer () from
/etc/httpd/modules/libphp4.so
#18 0x01121790 in php_end_ob_buffers () from
/etc/httpd/modules/libphp4.so
#19 0x011108e8 in php_request_shutdown () from
/etc/httpd/modules/libphp4.so
#20 0x01150443 in execute () from
/etc/httpd/modules/libphp4.so
#21 0x011506a8 in execute () from
/etc/httpd/modules/libphp4.so
#22 0x0807483c in ap_pass_brigade (next=0x7531,
bb=0x1)
at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/server/util_filter.c:541
#23 0x0807c489 in default_handler (r=0x11f1818)
at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/server/core.c:3555
#24 0x080685c5 in ap_run_handler (r=0xa045960)
at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/server/config.c:200
#25 0x08068bdf in ap_invoke_handler (r=0xa045960)
at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/server/config.c:410
#26 0x08065266 in rename_original_env (p=0xa045960,
t=0xfffffffc)
at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/modules/http/http_request.c:317
#27 0x080608bc in ap_process_http_connection
(c=0x99450c8)
at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/modules/http/http_core.c:293
#28 0x08072195 in ap_run_process_connection
(c=0x9a80ef8)
at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/server/connection.c:85
#29 0x08066ae1 in child_main (child_num_arg=-4)
at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/server/mpm/prefork/prefork.c:706
#30 0x08066c34 in make_child (s=0x7531, slot=6)
at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/server/mpm/prefork/prefork.c:749
#31 0x08066d56 in perform_idle_server_maintenance
(p=0x8)
at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/server/mpm/prefork/prefork.c:841
#32 0x0806757d in ap_mpm_run (_pconf=0x97bf0a8,
plog=0x97e9150, s=0x97bf0a8)
at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/server/mpm/prefork/prefork.c:983
#33 0x0806da4f in main (argc=1, argv=0xbfffb614)
at
/usr/local/src/sdudley/RPM/BUILD/httpd-2.0.46/server/main.c:671
==
Also, in case it matters, here is what the scoreboard
looks like when the system gets wedged. We have
plenty of memory and CPU cycles left when this
happens.
Apache Server Status for example.com
Server Version: Apache
Server Built: Feb 25 2005 09:35:45
Current Time: Thursday, 08-Sep-2005 15:01:42 CDT
Restart Time: Thursday, 08-Sep-2005 14:40:49 CDT
Parent Server Generation: 0
Server uptime: 20 minutes 53 seconds
Total accesses: 15303 - Total Traffic: 163.1 MB
CPU Usage: u869.65 s79.58 cu.29 cs.44 - 75.8% CPU load
12.2 requests/sec - 133.3 kB/second - 10.9 kB/request
150 requests currently being processed, 0 idle workers
WWWWWWWWWWWWWWWWWWWWWWCWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW
WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW
WWWWWWWWWWWWWWWWWWWWWW..........................................
................................................................
Scoreboard Key:
"_" Waiting for Connection, "S" Starting up, "R"
Reading Request,
"W" Sending Reply, "K" Keepalive (read), "D" DNS
Lookup,
"C" Closing connection, "L" Logging, "G" Gracefully
finishing,
"I" Idle cleanup of worker, "." Open slot with no
current process
Srv PID Acc M CPU SS Req Conn Child Slot
Client VHost
Request
0-0 14767 0/140/554 W 9.24 60 0 0.0 1.40
5.57
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
1-0 14861 0/4/126 W 0.83 19 0 0.0 0.08 2.33
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
2-0 14092 0/49/49 W 18.10 1036 0 0.0 0.35 0.35
65.162.244.194 example.com GET
/projects/viewProject.php?_mode=search&_clearFlag=course,sp
3-0 14862 0/4/556 W 1.03 20 0 0.0 0.10 4.59
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
4-0 14094 0/605/605 W 71.52 55 0 0.0 7.80
7.80
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
5-0 14386 0/541/548 W 36.02 46 0 0.0 5.89
5.90
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
6-0 14863 0/2/53 W 0.63 30 0 0.0 0.05 0.75
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
7-0 14097 0/260/260 W 26.50 48 0 0.0 2.20
2.20
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
8-0 14098 0/216/216 W 37.65 6 0 0.0 4.00
4.00
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
9-0 14589 0/356/524 W 39.67 55 0 0.0 3.79
5.60
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
10-0 14864 0/5/169 W 0.73 45 0 0.0 0.05 1.83
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
11-0 14218 0/632/638 W 58.11 0 0 0.0 7.82
7.88
66.190.215.152 example.com GET / HTTP/1.1
12-0 14102 0/47/47 W 3.44 49 0 0.0 0.60 0.60
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
13-0 14865 0/3/138 W 0.44 61 0 0.0 0.18 1.64
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
14-0 14866 0/5/249 W 0.78 3 0 0.0 0.08 4.70
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
15-0 14105 0/608/608 W 60.14 13 0 0.0 6.56
6.56
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
16-0 14280 0/583/587 W 56.92 44 0 0.0 5.10
5.12
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
17-0 14867 0/3/22 W 0.65 31 0 0.0 0.06 0.46
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
18-0 14868 0/5/105 W 1.72 0 0 0.0 0.12 0.98
209.223.94.229 example.com GET /voodoo-status HTTP/1.0
19-0 14283 0/565/575 W 51.57 52 0 0.0 5.97
6.03
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
20-0 14869 0/2/173 W 0.54 8 0 0.0 0.05 1.60
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
21-0 14111 0/668/668 W 49.32 36 0 0.0 5.97
5.97
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
22-0 14870 1/11/20 C 1.89 0 0 11.4 0.16 0.33
66.83.74.234 example.com GET
/uploaded_files/deliverFile.php/hon%20icon.jpg
HTTP/1.1
23-0 14871 0/6/8 W 1.02 43 0 0.0 0.06 0.07
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
24-0 14872 0/3/4 W 0.35 53 0 0.0 0.03 0.04
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
25-0 14873 0/1/371 W 0.29 56 0 0.0 0.03 4.66
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
26-0 14874 0/2/14 W 0.58 5 0 0.0 0.05 0.10
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
27-0 14875 0/4/15 W 1.06 0 0 0.0 0.07 0.17
66.18.105.194 example.com GET /r.php/1120902432
HTTP/1.1
28-0 14876 0/5/6 W 0.53 53 0 0.0 0.25 0.25
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
29-0 14877 0/1/4 W 0.31 46 0 0.0 0.03 0.13
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
30-0 14129 0/632/632 W 58.34 59 0 0.0 6.87
6.87
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
31-0 14878 0/5/104 W 0.63 43 0 0.0 0.05 1.35
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
32-0 14879 0/3/10 W 0.42 47 0 0.0 0.04 0.06
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
33-0 14880 0/4/12 W 0.92 39 0 0.0 0.08 0.14
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
34-0 14881 0/2/19 W 0.65 42 0 0.0 0.05 0.13
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
35-0 14138 0/81/81 W 5.14 56 0 0.0 1.04 1.04
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
36-0 14882 0/2/14 W 0.42 49 0 0.0 0.04 0.30
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
37-0 14883 0/1/18 W 0.30 50 0 0.0 0.03 0.07
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
38-0 14145 0/669/669 W 50.81 47 0 0.0 6.89
6.89
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
39-0 14884 0/2/8 W 0.40 45 0 0.0 0.18 0.22
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
40-0 14885 0/5/111 W 0.77 37 0 0.0 0.05 1.21
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
41-0 14886 0/6/30 W 0.54 50 0 0.0 0.06 0.27
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
42-0 14149 0/1/1 W 0.00 1235 0 0.0 0.0000 0.0000
64.132.138.66 example.com POST /login/login.php
HTTP/1.1
43-0 14150 0/634/634 W 56.34 57 0 0.0 5.24
5.24
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
44-0 14887 0/3/29 W 0.61 41 0 0.0 0.05 0.23
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
45-0 14888 0/2/6 W 0.51 39 0 0.0 0.04 0.10
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
46-0 14154 0/0/0 W 0.00 1232 0 0.0 0.00 0.00
64.132.138.66 example.com POST /login/login.php
HTTP/1.1
47-0 14155 0/253/253 W 14.58 38 0 0.0 2.02
2.02
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
48-0 14889 0/2/117 W 0.43 51 0 0.0 0.04 0.95
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
49-0 14890 0/2/11 W 0.34 52 0 0.0 0.03 0.06
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
50-0 14891 0/2/22 W 0.34 47 0 0.0 0.03 0.12
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
51-0 14894 0/1/59 W 0.32 35 0 0.0 0.03 0.81
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
52-0 14895 0/1/328 W 0.30 33 0 0.0 0.03 2.40
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
53-0 14896 0/2/46 W 0.32 32 0 0.0 0.03 0.54
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
54-0 14897 0/1/20 W 0.29 28 0 0.0 0.03 0.22
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
55-0 14898 0/1/10 W 0.30 26 0 0.0 0.03 0.10
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
56-0 14899 0/1/196 W 0.33 27 0 0.0 0.03 2.12
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
57-0 14900 0/1/328 W 0.27 30 0 0.0 0.03 2.98
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
58-0 14901 0/1/89 W 0.28 21 0 0.0 0.03 0.85
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
59-0 14902 0/1/22 W 0.35 18 0 0.0 0.03 0.10
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
60-0 14168 0/598/598 W 54.77 54 0 0.0 7.50
7.50
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
61-0 14903 0/1/124 W 0.33 24 0 0.0 0.03 1.94
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
62-0 14173 0/713/713 W 59.91 38 0 0.0 7.55
7.55
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
63-0 14904 0/2/11 W 0.43 17 0 0.0 0.05 0.07
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
64-0 14905 0/1/6 W 0.33 22 0 0.0 0.03 0.05
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
65-0 14906 0/1/104 W 0.28 24 0 0.0 0.03 0.85
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
66-0 14178 0/425/425 W 41.45 51 0 0.0 4.58
4.58
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
67-0 14907 0/0/5 W 0.42 71 0 0.0 0.00 0.05
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
68-0 14908 0/1/2 W 0.32 25 0 0.0 0.03 0.05
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
69-0 14909 0/2/87 W 0.37 10 0 0.0 0.03 0.95
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
70-0 14910 0/1/7 W 0.35 16 0 0.0 0.03 0.04
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
71-0 14911 0/2/64 W 0.61 5 0 0.0 0.04 0.91
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
72-0 14912 0/1/154 W 0.28 14 0 0.0 0.03 1.10
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
73-0 14913 0/1/483 W 0.33 12 0 0.0 0.03 4.37
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
74-0 14186 0/592/592 W 44.76 9 0 0.0 5.73
5.73
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
75-0 14914 0/1/1 W 0.31 11 0 0.0 0.03 0.03
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
76-0 14915 0/0/0 W 0.00 70 0 0.0 0.00 0.00
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
77-0 14916 0/1/1 W 0.35 10 0 0.0 0.03 0.03
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
78-0 14917 0/1/1 W 0.28 9 0 0.0 0.03 0.03
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
79-0 14918 0/0/0 W 0.00 70 0 0.0 0.00 0.00
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
80-0 14919 0/7/7 W 0.72 42 0 0.0 0.07 0.07
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
81-0 14920 0/1/1 W 0.29 7 0 0.0 0.03 0.03
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
82-0 14921 0/1/1 W 0.27 7 0 0.0 0.03 0.03
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
83-0 14922 0/1/1 W 0.33 8 0 0.0 0.03 0.03
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
84-0 14923 0/6/6 W 1.06 1 0 0.0 0.11 0.11
139.76.128.68 example.com GET
/uploaded_files/deliverFile.php/playground%20hurricane.gif
85-0 14924 0/1/1 W 0.34 6 0 0.0 0.03 0.03
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
86-0 14925 0/2/2 W 0.33 13 0 0.0 0.03 0.03
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
87-0 14926 0/3/3 W 0.43 31 0 0.0 0.03 0.03
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
88-0 14927 0/2/2 W 0.31 14 0 0.0 0.03 0.03
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
89-0 14928 0/5/5 W 0.48 29 0 0.0 0.03 0.03
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
90-0 14929 0/1/1 W 0.32 29 0 0.0 0.03 0.03
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
91-0 14930 0/1/1 W 0.32 11 0 0.0 0.03 0.03
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
92-0 14931 0/1/1 W 0.27 28 0 0.0 0.03 0.03
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
93-0 14932 0/1/1 W 0.32 26 0 0.0 0.03 0.03
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
94-0 14933 0/1/1 W 0.31 27 0 0.0 0.03 0.03
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
95-0 14934 0/2/2 W 0.33 16 0 0.0 0.03 0.03
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
96-0 14935 0/6/6 W 0.49 34 0 0.0 0.07 0.07
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
97-0 14936 0/1/1 W 0.27 19 0 0.0 0.03 0.03
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
98-0 14937 0/6/6 W 0.56 43 0 0.0 0.04 0.04
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
99-0 14938 0/8/8 W 0.44 41 0 0.0 0.03 0.03
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
100-0 14939 0/4/4 W 0.10 67 0 0.0 0.03 0.03
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
101-0 14940 0/2/2 W 0.28 17 0 0.0 0.03 0.03
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
102-0 14941 0/1/1 W 0.31 25 0 0.0 0.03 0.03
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
103-0 14942 0/6/6 W 0.70 37 0 0.0 0.06 0.06
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
104-0 14943 0/5/5 W 0.44 34 0 0.0 0.03 0.03
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
105-0 14944 0/3/3 W 0.34 15 0 0.0 0.03 0.03
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
106-0 14945 0/4/4 W 0.53 32 0 0.0 0.05 0.05
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
107-0 14946 0/2/2 W 0.28 22 0 0.0 0.03 0.03
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
108-0 14947 0/1/1 W 0.29 15 0 0.0 0.03 0.03
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
109-0 14948 0/1/1 W 0.30 23 0 0.0 0.03 0.03
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
110-0 14949 0/1/1 W 0.28 21 0 0.0 0.03 0.03
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
111-0 14950 0/4/4 W 0.39 35 0 0.0 0.04 0.04
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
112-0 14951 0/7/7 W 0.71 36 0 0.0 0.05 0.05
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
113-0 14952 0/2/2 W 0.36 18 0 0.0 0.03 0.03
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
114-0 14953 0/1/1 W 0.35 20 0 0.0 0.03 0.03
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
115-0 14954 0/2/2 W 0.43 33 0 0.0 0.04 0.04
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
116-0 14955 0/5/5 W 0.44 39 0 0.0 0.05 0.05
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
117-0 14956 0/5/5 W 0.67 4 0 0.0 0.04 0.04
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
118-0 14957 0/5/5 W 0.09 65 0 0.0 0.01 0.01
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
119-0 14958 0/3/3 W 0.64 2 0 0.0 0.05 0.05
152.130.8.6 example.com GET
/projects/viewProject.php?_mode=occurrenceView&_action=load
120-0 14959 0/1/1 W 0.42 0 0 0.0 0.03 0.03
65.87.105.1 example.com GET
/projects/viewProject.php?_Offset=10&_mode=browse&_action=f
121-0 14960 0/0/0 W 0.00 64 0 0.0 0.00 0.00
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
122-0 14961 0/0/0 W 0.00 64 0 0.0 0.00 0.00
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
123-0 14962 0/0/0 W 0.00 64 0 0.0 0.00 0.00
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
124-0 14963 0/1/1 W 0.00 64 0 0.0 0.0000 0.0000
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
125-0 14964 0/0/0 W 0.00 63 0 0.0 0.00 0.00
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
126-0 14965 0/0/0 W 0.00 63 0 0.0 0.00 0.00
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
127-0 14966 0/4/4 W 0.34 62 0 0.0 0.20 0.20
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
128-0 14967 0/1/1 W 0.01 63 0 0.0 0.0000 0.0000
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
129-0 14968 0/0/0 W 0.00 63 0 0.0 0.00 0.00
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
130-0 14969 0/0/0 W 0.00 63 0 0.0 0.00 0.00
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
131-0 14970 0/1/1 W 0.00 63 0 0.0 0.00 0.00
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
132-0 14971 0/2/2 W 0.28 62 0 0.0 0.02 0.02
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
133-0 14972 0/0/0 W 0.00 62 0 0.0 0.00 0.00
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
134-0 14973 0/1/1 W 0.00 62 0 0.0 0.00 0.00
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
135-0 14974 0/0/0 W 0.00 62 0 0.0 0.00 0.00
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
136-0 14975 0/0/0 W 0.00 62 0 0.0 0.00 0.00
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
137-0 14976 0/0/0 W 0.00 62 0 0.0 0.00 0.00
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
138-0 14977 0/0/0 W 0.00 62 0 0.0 0.00 0.00
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
139-0 14978 0/0/0 W 0.00 62 0 0.0 0.00 0.00
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
140-0 14979 0/0/0 W 0.00 62 0 0.0 0.00 0.00
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
141-0 14980 0/0/0 W 0.00 62 0 0.0 0.00 0.00
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
142-0 14981 0/1/1 W 0.00 62 0 0.0 0.00 0.00
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
143-0 14982 0/0/0 W 0.00 62 0 0.0 0.00 0.00
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
144-0 14983 0/0/0 W 0.00 62 0 0.0 0.00 0.00
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
145-0 14984 0/0/0 W 0.00 62 0 0.0 0.00 0.00
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
146-0 14985 0/0/0 W 0.00 62 0 0.0 0.00 0.00
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
147-0 14986 0/1/1 W 0.08 61 0 0.0 0.00 0.00
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
148-0 14987 0/0/0 W 0.00 62 0 0.0 0.00 0.00
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
149-0 14988 0/0/0 W 0.00 61 0 0.0 0.00 0.00
65.168.18.47 example.com GET
/projects/viewProject.php?_Offset=10&_mode=occurrenceView&_
Srv Child Server number - generation
PID OS process ID
Acc Number of accesses this connection / this child /
this slot
M Mode of operation
CPU CPU usage, number of seconds
SS Seconds since beginning of most recent request
Req Milliseconds required to process most recent
request
Conn Kilobytes transferred this connection
Child Megabytes transferred this child
Slot Total megabytes transferred this slot
Apache Server at www.example.com Port 80
Thanks!
SD
______________________________________________________
Click here to donate to the Hurricane Katrina relief effort.
http://store.yahoo.com/redcross-donate3/
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: [EMAIL PROTECTED]
" from the digest: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
