Otto Moerbeek wrote:
On Wed, 11 Jul 2007, Adam PAPAI wrote:

Hello,

I have the following problem with the php and the httpd.
We have a bit-modified kernel:

in /usr/src/sys/arch/i386/conf/GENERIC we changed the
maxusers    256


unaame -a
OpenBSD arsenic.digitalinfluence.hu 4.1 GENERIC.MP#1 i386


The system hangs after 1 day. It's a very strongly loaded system. Any idea
why? It seems that the system does not really handle the http requests well.

/var/www/log/access_log snippet:

httpd in malloc(): error: recursive call
httpd in malloc(): error: recursive call
[Mon Jul  9 18:33:39 2007] [notice] child pid 24755 exit signal Abort (6)
[Mon Jul  9 18:33:39 2007] [notice] child pid 10372 exit signal Abort (6)
[Mon Jul  9 18:33:39 2007] [notice] child pid 11192 exit signal Segmentation
fault (11)
[Mon Jul  9 18:33:39 2007] [notice] child pid 15994 exit signal Segmentation
fault (11)
[Mon Jul  9 18:33:42 2007] [error] [client 84.3.75.29] File does not exist:
/users/reklamcsere/favicon.ico
[Mon Jul  9 18:33:53 2007] [error] [client 83.216.56.20] File does not exist:
/users/reklamcsere/images/px1.gif

The aborts and seg faults are likely caused by php. It's mostly sloppy
programming by the php developers.

Why your system hangs is hard to tell. Apart from a quite high load
avarage I see no obvious cases of running out of resources. vmstat -m
might give a clue.  You could be also running out of file descriptors.
Try increasing kern.maxfiles.

A know problem exists with PHP and anonymous mmaps in the kernel. I do not know if this is what hits you here, but the symptons look much like it.

a busy system that runs, e.g. the horde/imp webmailer will hang within a short time (1.5 days over here).

until there is a better solution, the workaround is quite easy: restart httpd in a cronjob.


last load:

load averages: 166.08, 190.05, 175.78
18:33:49
238 processes: 1 running, 236 idle, 1 on processor
CPU0 states:  8.8% user,  0.0% nice,  6.0% system,  0.0% interrupt, 85.2% idle
CPU1 states:  0.8% user,  0.0% nice,  1.4% system,  0.0% interrupt, 97.8% idle
Memory: Real: 213M/511M act/tot  Free: 491M  Swap: 0K/512M used/tot

  PID USERNAME PRI NICE  SIZE   RES STATE    WAIT     TIME    CPU COMMAND
  16469 www        2    0 3432K 4724K sleep/0  netio    1:01  1.61% httpd
  11982 www        2    0 3424K 4724K sleep/0  netio    0:21  1.51% httpd
  27986 www        2    0 3408K 4672K sleep/0  netio    0:19  1.51% httpd
  29148 www        2    0 3824K 4952K sleep/0  netio    1:00  1.46% httpd
  23427 www        2    0 3432K 4724K sleep/0  netio    0:23  1.42% httpd
   1392 www        2    0 3428K 4876K sleep/0  netio    0:45  1.37% httpd
   16673 www        2    0 3428K 4756K sleep/0  netio    1:40  1.32% httpd
   10830 www        2    0 3420K 4724K sleep/1  netio    1:22  1.32% httpd
   32318 www        2    0 3408K 4616K sleep/0  netio    1:07  1.32% httpd


vmstat:

 4 users    Load166.08190.05175.78                  Mon Jul  9 18:33:49 2007
            memory totals (in KB)            PAGING   SWAPPING Interrupts
           real   virtual     free           in  out   in  out      773 total
Active   218444    218444   502752   ops                            200 clock
All      523712    523712  1027036   pages                          163 ipi

 ahc0
Proc:r  d  s  w    Csw   Trp   Sys   Int   Sof  Flt     1 forks     201 fxp0
        3207       479  2141  3093   410   223 1933       fkppw     209 fxp1
                                                         fksvm
   0.0%Int   3.7%Sys   4.8%Usr   0.0%Nic  91.5%Idle       pwait
|    |    |    |    |    |    |    |    |    |    |       relck
==>>                                                      rlkok
                                                          noram
Namei         Sys-cache    Proc-cache    No-cache     324 ndcpy
    Calls     hits    %    hits     %    miss   %         fltcp
     2047     2047  100                              1107 zfod
                                                      103 cow
Disks   sd0   sd1   cd0   fd0                         128 fmin
seeks                                                 170 ftarg
xfers                                               26444 itarg
Kbyte    13                                             3 wired
  sec   0.0                                               pdfre
                                                          pdscn
                                                          pzidle
                                                     1308 kmapent


dmesg:
OpenBSD 4.1-current (GENERIC.MP) #1: Sun Jul  8 23:42:42 CEST 2007

[EMAIL PROTECTED]:/usr/src/sys/arch/i386/compile/GENERIC.MP
cpu0: Intel Pentium III ("GenuineIntel" 686-class) 1 GHz
cpu0:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE
real mem  = 1073221632 (1048068K)
avail mem = 971218944 (948456K)
using 4278 buffers containing 53784576 bytes (52524K) of memory
mainbus0 (root)
bios0 at mainbus0: AT/286+ BIOS, date 05/15/02, BIOS32 rev. 0 @ 0xfd5e1
pcibios0 at bios0: rev 2.1 @ 0xf0000/0xffff
pcibios0: PCI BIOS has 7 Interrupt Routing table entries
pcibios0: PCI Interrupt Router at 000:15:0 ("ServerWorks OSB4" rev 0x00)
pcibios0: PCI bus #0 is the last bus
bios0: ROM list: 0xc0000/0xa000 0xca000/0x1800 0xcb800/0x5200
acpi at mainbus0 not configured
mainbus0: Intel MP Specification (Version 1.4)
cpu0 at mainbus0: apid 3 (boot processor)
cpu0: apic clock running at 132 MHz
cpu1 at mainbus0: apid 0 (application processor)
cpu1: Intel Pentium III ("GenuineIntel" 686-class)
cpu1: FPU,CX8,APIC
mainbus0: bus 0 is type PCI
mainbus0: bus 1 is type PCI
mainbus0: bus 2 is type ISA
ioapic0 at mainbus0: apid 14 pa 0xfec00000, version 11, 16 pins
ioapic1 at mainbus0: apid 13 pa 0xfec01000, version 11, 16 pins
pci0 at mainbus0 bus 0: configuration mode 1 (no bios)
pchb0 at pci0 dev 0 function 0 "ServerWorks CNB20LE Host" rev 0x06
pchb1 at pci0 dev 0 function 1 "ServerWorks CNB20LE Host" rev 0x06
pci1 at pchb1 bus 1
ahc0 at pci1 dev 3 function 0 "Adaptec AIC-7892 U160" rev 0x02: apic 13 int 12
(irq 9)
scsibus0 at ahc0: 16 targets
ahc0: target 0 using 8bit transfers
ahc0: target 0 using asynchronous transfers
sd0 at scsibus0 targ 0 lun 0: <IBM-ESXS, ST336605LC !#, B247> SCSI3 0/direct
fixed
sd0: 34715MB, 29550 cyl, 4 head, 601 sec, 512 bytes/sec, 71096640 sec total
ahc0: target 1 using 8bit transfers
ahc0: target 1 using asynchronous transfers
sd1 at scsibus0 targ 1 lun 0: <IBM-PSG, ST39175LC !#, 0430> SCSI2 0/direct
fixed
sd1: 8678MB, 11721 cyl, 5 head, 303 sec, 512 bytes/sec, 17774160 sec total
safte0 at scsibus0 targ 8 lun 0: <IBM, FTlV1 S2, 0> SCSI2 3/processor fixed
vga1 at pci0 dev 1 function 0 "S3 Savage 4" rev 0x04
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
fxp0 at pci0 dev 2 function 0 "Intel 8255x" rev 0x08, i82559: apic 13 int 11
(irq 10), address 00:02:55:54:52:8e
inphy0 at fxp0 phy 1: i82555 10/100 PHY, rev. 4
fxp1 at pci0 dev 10 function 0 "Intel 8255x" rev 0x08, i82559: apic 13 int 9
(irq 10), address 00:02:55:54:52:8f
inphy1 at fxp1 phy 1: i82555 10/100 PHY, rev. 4
piixpm0 at pci0 dev 15 function 0 "ServerWorks OSB4" rev 0x50: SMBus disabled
pciide0 at pci0 dev 15 function 1 "ServerWorks OSB4 IDE" rev 0x00: DMA
atapiscsi0 at pciide0 channel 0 drive 0
scsibus1 at atapiscsi0: 2 targets
cd0 at scsibus1 targ 0 lun 0: <TEAC, CD-224E, 2.7B> SCSI0 5/cdrom removable
cd0(pciide0:0:0): using PIO mode 4, DMA mode 2, Ultra-DMA mode 2
ohci0 at pci0 dev 15 function 2 "ServerWorks OSB4/CSB5 USB" rev 0x04: apic 14
int 7 (irq 7), version 1.0, legacy support
usb0 at ohci0: USB revision 1.0
uhub0 at usb0
uhub0: ServerWorks OHCI root hub, rev 1.00/1.00, addr 1
uhub0: 2 ports with 2 removable, self powered
isa0 at mainbus0
isadma0 at isa0
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pcppi0 at isa0 port 0x61
midi0 at pcppi0: <PC speaker>
spkr0 at pcppi0
npx0 at isa0 port 0xf0/16: reported by CPUID; using exception 16
pccom2 at isa0 port 0x3e8/8 irq 5: ns16550a, 16 byte fifo
fdc0 at isa0 port 0x3f0/6 irq 6 drq 2
fd0 at fdc0 drive 0: 1.44MB 80 cyl, 2 head, 18 sec
pctr: 686-class user-level performance counters enabled
mtrr: Pentium Pro MTRR support
ahc0: target 0 using 16bit transfers
ahc0: target 0 synchronous at 80.0MHz DT, offset = 0x3f
dkcsum: sd0 matches BIOS drive 0x80
ahc0: target 1 using 16bit transfers
ahc0: target 1 synchronous at 40.0MHz, offset = 0xf
dkcsum: sd1 matches BIOS drive 0x81
root on sd0a
rootdev=0x400 rrootdev=0xd00 rawdev=0xd02
WARNING: / was not properly unmounted
arp: attempt to add entry for 10.5.5.5 on fxp0 by 00:03:ba:11:db:58 on fxp1
arsenic[4.1][wooh]>

Thanks in advance



--
Adam PAPAI
D i g i t a l Influence
http://www.digitalinfluence.hu
E-mail: [EMAIL PROTECTED]
Phone: +36 30 33-55-735 (Hungary)

Reply via email to