SMP problems post Jan 28th

[Mike Tancsa]

The previous kernel had been running fine for some time, and now at night around the running of periodic, the box will 'periodically' crash. It does not seem to do it each night, but almost every other night and always just after 3am when periodic runs. I can never get a crash dump, and I had to hook up a serial console to capture this at night. Also, I cant seem to force the issue by running periodic by hand. But, like I said, it always seems to happen a few minutes after 3am. (No, nothing else is scheduled to run then and no other boxes do anything to it at that time either)


Fatal trap 12: page fault while in kernel mode
mp_lock = 01000002; cpuid = 1; lapic.id = 00000000
fault virtual address = 0x65b046a5
fault code = supervisor read, page not present
instruction pointer = 0x8:0xc0174830
stack pointer = 0x10:0xde174c4c
frame pointer = 0x10:0xde174c58
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, def32 1, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 4203 (find)
interrupt mask = none <- SMP: XXX
trap number = 12
panic: page fault
mp_lock = 01000002; cpuid = 1; lapic.id = 00000000
boot() called on cpu#1

syncing disks... 4 2
done
Uptime: 1d15h8m8s
Automatic reboot in 15 seconds - press a key on the console to abort
Rebooting...
cpu_reset called on cpu#1
cpu_reset: Stopping other CPUs
cpu_reset: Restarting BSP
tpu_reset_proxy: Grabbed mp lock cfpour_reSsPe
: BSP did not grab mp lock
Console: serial port
BIOS drive A: is disk0
BIOS drive C: is disk1
BIOS 629kB/785396kB available memory


Any ideas how best to track this down ? There seem to been some commits on the 30th that might have had an effect.

4.7-STABLE FreeBSD 4.7-STABLE #0: Thu Feb 6 06:04:02 EST 2003

ns4# dmesg
Copyright (c) 1992-2003 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD 4.7-STABLE #0: Thu Feb 6 06:04:02 EST 2003
[EMAIL PROTECTED]:/usr/obj/usr/src/sys/smp
Timecounter "i8254" frequency 1193182 Hz
CPU: Intel Pentium III (801.82-MHz 686-class CPU)
Origin = "GenuineIntel" Id = 0x683 Stepping = 3
Features=0x383fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE>
real memory = 805294080 (786420K bytes)
config> q
avail memory = 779272192 (761008K bytes)
Programming 24 pins in IOAPIC #0
IOAPIC #0 intpin 2 -> irq 0
IOAPIC #0 intpin 17 -> irq 11
IOAPIC #0 intpin 18 -> irq 10
IOAPIC #0 intpin 19 -> irq 12
FreeBSD/SMP: Multiprocessor motherboard
cpu0 (BSP): apic id: 1, version: 0x00040011, at 0xfee00000
cpu1 (AP): apic id: 0, version: 0x00040011, at 0xfee00000
io0 (APIC): apic id: 2, version: 0x00170011, at 0xfec00000
Preloaded elf kernel "kernel" at 0xc03b5000.
Preloaded userconfig_script "/boot/kernel.conf" at 0xc03b509c.
Pentium Pro MTRR support enabled
md0: Malloc disk
Using $PIR table, 6 entries at 0xc00f0d20
npx0: <math processor> on motherboard
npx0: INT 16 interface
pcib0: <Intel 82443BX (440 BX) host to PCI bridge> on motherboard
pci0: <PCI bus> on pcib0
pcib1: <Intel 82443BX (440 BX) PCI-PCI (AGP) bridge> at device 1.0 on pci0
pci1: <PCI bus> on pcib1
pci1: <ATI Mach64-GB graphics accelerator> at 0.0
isab0: <Intel 82371AB PCI to ISA bridge> at device 4.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel PIIX4 ATA33 controller> port 0xb800-0xb80f at device 4.1 on pci0
ata0: at 0x1f0 irq 14 on atapci0
ata1: at 0x170 irq 15 on atapci0
pci0: <Intel 82371AB/EB (PIIX4) USB controller> at 4.2
Timecounter "PIIX" frequency 3579545 Hz
chip1: <Intel 82371AB Power management controller> port 0xe800-0xe80f at device 4.3 on pci0
fxp0: <Intel Pro 10/100B/100+ Ethernet> port 0xb000-0xb03f mem 0xe3800000-0xe38fffff,0xe4000000-0xe4000fff irq 12 at device 9.0 on pci0
fxp0: Ethernet address 00:02:b3:07:fd:8d
inphy0: <i82555 10/100 media interface> on miibus0
inphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
fxp1: <Intel Pro 10/100B/100+ Ethernet> port 0xa800-0xa81f mem 0xe3000000-0xe30fffff,0xe6800000-0xe6800fff irq 10 at device 10.0 on pci0
fxp1: Ethernet address 00:a0:c9:e7:a6:e6
inphy1: <i82555 10/100 media interface> on miibus1
inphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
twe0: <3ware Storage Controller> port 0xa400-0xa40f irq 11 at device 11.0 on pci0
twe0: 2 ports, Firmware FE6X 1.02.00.029, BIOS BEXX 1.07.00.009
orm0: <Option ROMs> at iomem 0xc0000-0xc7fff,0xc8000-0xc8fff,0xcc000-0xccfff on isa0
fdc0: <NEC 72065B or clone> at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on isa0
fdc0: FIFO enabled, 8 bytes threshold
fd0: <1440-KB 3.5" drive> on fdc0 drive 0
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x100>
sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0
sio0: type 16550A, console
sio1 at port 0x2f8-0x2ff irq 3 on isa0
sio1: type 16550A
ppc0: <Parallel port> at port 0x378-0x37f irq 7 on isa0
ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode
ppc0: FIFO with 16/16/9 bytes threshold
ppi0: <Parallel I/O> on ppbus0
plip0: <PLIP network interface> on ppbus0
lpt0: <Printer> on ppbus0
lpt0: Interrupt-driven port
APIC_IO: Testing 8254 interrupt delivery
APIC_IO: routing 8254 via IOAPIC #0 intpin 2
DUMMYNET initialized (011031)
ipfw2 initialized, divert enabled, rule-based forwarding enabled, default to accept, logging limited to 34100 packets/entry by default
IPv6 packet filtering initialized, default to accept, logging limited to 100 packets/entry
twed0: <TwinStor, Normal> on twe0
twed0: 76318MB (156299440 sectors)
Mounting root from ufs:/dev/twed0a
SMP: AP CPU #1 Launched!
WARNING: / was not properly dismounted
ns4#


config is pretty simple

grep -v ^# smp | strings
machine i386
cpu I586_CPU
cpu I686_CPU
ident smp
maxusers 0
makeoptions DEBUG=-g #Build kernel with gdb(1) debug symbols
options MATH_EMULATE #Support for x87 emulation
options INET #InterNETworking
options INET6 #IPv6 communications protocols
options FFS #Berkeley Fast Filesystem
options FFS_ROOT #FFS usable as root device [keep this!]
options SOFTUPDATES #Enable FFS soft updates support
options UFS_DIRHASH #Improve performance on big directories
options MFS #Memory Filesystem
options MD_ROOT #MD is a potential root device
options NFS #Network Filesystem
options NFS_ROOT #NFS usable as root device, NFS required
options PROCFS #Process filesystem
options COMPAT_43 #Compatible with BSD 4.3 [KEEP THIS!]
options SCSI_DELAY=15000 #Delay (in ms) before probing SCSI
options UCONSOLE #Allow users to grab the console
options USERCONFIG #boot -c editor
options VISUAL_USERCONFIG #visual boot -c editor
options KTRACE #ktrace(1) support
options SYSVSHM #SYSV-style shared memory
options SYSVMSG #SYSV-style message queues
options SYSVSEM #SYSV-style semaphores
options P1003_1B #Posix P1003_1B real-time extensions
options _KPOSIX_PRIORITY_SCHEDULING
options ICMP_BANDLIM #Rate limit bad replies
options KBD_INSTALL_CDEV # install a CDEV entry in /dev
# output. Adds ~128k to driver.
# output. Adds ~215k to driver.
options SMP # Symmetric MultiProcessor Kernel
options APIC_IO # Symmetric (APIC) I/O
device isa
device pci
device fdc0 at isa? port IO_FD1 irq 6 drq 2
device fd0 at fdc0 drive 0
device ata
device atadisk # ATA disk drives
device atapicd # ATAPI CDROM drives
device atapist # ATAPI tape drives
options ATA_STATIC_ID #Static device numbering
device ahc # AHA2940 and onboard AIC7xxx devices
# Allow ncr to attach legacy NCR devices when
# both sym and ncr are configured
device scbus # SCSI bus (required)
device da # Direct Access (disks)
device sa # Sequential Access (tape etc)
device cd # CD
device pass # Passthrough device (direct SCSI access)
device twe # 3ware Escalade
device atkbdc0 at isa? port IO_KBD
device atkbd0 at atkbdc? irq 1 flags 0x1
device psm0 at atkbdc? irq 12
device vga0 at isa?
pseudo-device splash
device sc0 at isa? flags 0x100
device npx0 at nexus? port IO_NPX irq 13
device apm0 at nexus? disable flags 0x20 # Advanced Power Management
device sio0 at isa? port IO_COM1 flags 0x10 irq 4
device sio1 at isa? port IO_COM2 irq 3
device ppc0 at isa? irq 7
device ppbus # Parallel port bus (required)
device lpt # Printer
device plip # TCP/IP over parallel
device ppi # Parallel port interface device
device miibus # MII bus support
device dc # DEC/Intel 21143 and various workalikes
device fxp # Intel EtherExpress PRO/100B (82557, 82558)
device rl # RealTek 8129/8139
pseudo-device loop # Network loopback
pseudo-device ether # Ethernet support
pseudo-device ppp 1 # Kernel PPP
pseudo-device tun # Packet tunnel.
pseudo-device pty # Pseudo-ttys (telnet etc)
pseudo-device md # Memory "disks"
pseudo-device disc # Memory "disks"
pseudo-device gif # IPv6 and IPv4 tunneling
pseudo-device faith 1 # IPv6-to-IPv4 relaying (translation)
pseudo-device bpf #Berkeley packet filter
options IPFW2
options IPFIREWALL #firewall
options IPFIREWALL_VERBOSE #enable logging to syslogd(8)
options IPFIREWALL_FORWARD #enable transparent proxy support
options IPFIREWALL_VERBOSE_LIMIT=34100 #limit verbosity
options IPFIREWALL_DEFAULT_TO_ACCEPT #allow everything by default
options IPV6FIREWALL #firewall for IPv6
options IPV6FIREWALL_VERBOSE
options IPV6FIREWALL_VERBOSE_LIMIT=100
options IPV6FIREWALL_DEFAULT_TO_ACCEPT
options DUMMYNET
options IPDIVERT #divert sockets
options NETGRAPH #netgraph(4) system
options NETGRAPH_ASYNC
options NETGRAPH_BPF
options NETGRAPH_ECHO
options NETGRAPH_ETHER
options NETGRAPH_HOLE
options NETGRAPH_IFACE
options NETGRAPH_KSOCKET
options NETGRAPH_L2TP
options NETGRAPH_LMI
options NETGRAPH_MPPC_ENCRYPTION
options NETGRAPH_ONE2MANY
options NETGRAPH_PPP
options NETGRAPH_PPPOE
options NETGRAPH_PPTPGRE
options NETGRAPH_RFC1490
options NETGRAPH_SOCKET
options NETGRAPH_TEE
options NETGRAPH_TTY
options NETGRAPH_UI
options NETGRAPH_VJC
--------------------------------------------------------------------
Mike Tancsa, tel +1 519 651 3400
Sentex Communications, [EMAIL PROTECTED]
Providing Internet since 1994 www.sentex.net
Cambridge, Ontario Canada www.sentex.net/mike


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-stable" in the body of the message