On 31/01/12 16:39, Kapetanakis Giannis wrote:
Ok I'll try dmesg from ddb if that happens again.
Bad luck my dmesg is not saved through reboots, otherwise I would give
this info now...
thanx
Giannis
Hi,
I got this again after upgrading to 5.3 (two weeks ago) although I don't
think it's related since I'm getting
hungs from pimd since 5.0.
A second problem is that I couldn't do boot reboot from ddb and I had to
go there
to reset the machine.
Whole thread is in:
http://marc.info/?l=openbsd-misc&m=132800134306980&w=2
regards,
Giannis
uvm_fault(0xd0a977c0, 0xefffe000, 0, 1) -> d
kernel: page fault trap, code=0
Stopped at pf_find_state+0x54: movl 0x38(%eax),%edx
ddb{0}> PID PPID PGRP UID S FLAGS WAIT COMMAND
*18955 1 18955 0 7 0 pimd
ddb{0}> trace
pf_find_state(d43e7700,f554dae8,2,d942ba00,d9e91822) at pf_find_state+0x54
pf_test_state_udp(f554db64,f554db54,2,2,d43e7700) at pf_test_state_udp+0xbc
pf_test(2,2,d42df000,f554dcb4,0) at pf_test+0xbae
ip_output(d942ba00,0,0,1,f554dd30) at ip_output+0x54d
mrt_ioctl(d97bce00,0,3b9aca00,2,da094a00) at mrt_ioctl+0x1b6
mrt_ioctl(40,f554de4c,2c,d046062f,40) at mrt_ioctl+0xa4a
ip_mrouter_set(d9907724,68,f554def0,68,f554def0) at ip_mrouter_set+0x7c9
sosetopt(d9907724,0,68,d942b300,246) at sosetopt+0x50
sys_setsockopt(da06eba4,f554df64,f554df84,d069f594,1) at sys_setsockopt+0xc4
syscall() at syscall+0x120
--- syscall (number 7) ---
0x2:
ddb{0}>
pf_find_state(d43e7700,f554dae8,2,d942ba00,d9e91822) at pf_find_state+0x54
ddb{0}> show panic
the kernel did not panic
ddb{0}> machine ddbcpu 1
Stopped at Debugger+0x4: popl %ebp
ddb{1}> trace
Debugger(d409b800,d08a8046,1,200202,1) at Debugger+0x4
i386_ipi_handler(b0,d0430020,d0a60000,10,4d9d0010) at i386_ipi_handler+0x5f
Xintripi() at Xintripi+0x49
--- interrupt ---
__mp_lock(d0abb604,7fffffff,d9eef000,d0b769f4,f5554e1c) at __mp_lock+0x52
__mp_acquire_count(d0abb604,1,f5554e6c,d042f6ec,d0b769f4) at
__mp_acquire_count
+0x20
mi_switch(d0abfd3c,0,0,f5554e54,f5554e54) at mi_switch+0x1c2
sleep_finish(f5554e54,1,118,d093800f,c0) at sleep_finish+0x9e
tsleep(d0ac23e8,118,d093800f,0,f5554ea0) at tsleep+0x9b
doppoll(d9eef000,3c005260,a,0,0,f5554f84,f5554f1c,d042e172,d0abb604,286,f5554f1
c,0,f5554fa8) at doppoll+0x291
sys_poll(d9eef000,f5554f64,f5554f84,d069f594,1) at sys_poll+0x6f
syscall() at syscall+0x120
--- syscall (number 673172280) ---
0x2:
ddb{1}> machine ddbcpu 2
Stopped at Debugger+0x4: popl %ebp
ddb{2}> trace
Debugger(d409bc00,f5524f10,d07e64c1,0,d409bc34) at Debugger+0x4
i386_ipi_handler(0,d0430020,d0ab0000,10,d4090010) at i386_ipi_handler+0x5f
Xintripi() at Xintripi+0x49
--- interrupt ---
cpu_idle_cycle(d409bc00) at cpu_idle_cycle+0xf
Bad frame pointer: 0xd0c3fe28
ddb{2}> machine ddbcpu 3
Stopped at Debugger+0x4: popl %ebp
ddb{3}> trace
Debugger(d409c000,f5526f10,d07e64c1,0,d409c034) at Debugger+0x4
i386_ipi_handler(0,d0430020,d0ab0000,10,d4090010) at i386_ipi_handler+0x5f
Xintripi() at Xintripi+0x49
--- interrupt ---
cpu_idle_cycle(d409c000) at cpu_idle_cycle+0xf
Bad frame pointer: 0xd0c3fe28
ddb{3}> machine ddbcpu 4
Invalid cpu 4
ddb{3}> boot reboot
panic: kernel diagnostic assertion "p->p_wchan == NULL" failed: file
"../../../../kern/kern_sched.c", line 311
Stopped at Debugger+0x4: popl %ebp
ddb{3}> trace
Debugger(d0964e86,f5526a84,d09435b8,f5526a84,7fffffff) at Debugger+0x4
panic(d09435b8,d08b1dce,d094127c,d094124d,137) at panic+0x5d
__assert(d08b1dce,d094124d,137,d094127c,da0758b8) at __assert+0x2e
sched_chooseproc(da074858,da0758b8,f5526b48,d044a395,d409c034) at
sched_choosep
roc+0x175
mi_switch(d0abfd3c,d04487fc,da07d008,f5526b28,f5526b4c) at mi_switch+0x124
sleep_finish(f5526b4c,1,10,d090a584,d0a62380) at sleep_finish+0x9e
msleep(d40bc560,f5526b90,10,d090a584,0) at msleep+0x7b
mfi_exec(d40a9000,d40bc560,78,d0460713,0) at mfi_exec+0x94
mfi_do_mgmt(d40a9000,d40bc560,1101000,0,0) at mfi_do_mgmt+0xe9
mfi_scsi_cmd(d8af9440,d05eaf00,f5526c44,d40a8500,21) at mfi_scsi_cmd+0x1a5
scsi_xs_sync(d8af9440,3,f5526c88,0,d3f86c70) at scsi_xs_sync+0x55
sd_flush(d408b000,3,f5526cb8,d04373ad,d408b000) at sd_flush+0x5d
sd_shutdown(d408b000,1,0,4804,f5526d1c) at sd_shutdown+0x24
dohooks(d0a50464,1,f5526ce8,d0404905,d) at dohooks+0x6d
boot(4804,f5526d1c,f5526da8,d0402794,4) at boot+0x45
db_boot_reboot_cmd(4,0,ffffffff,f5526d20,0) at db_boot_reboot_cmd+0x12
db_command(d0a4e4c0,d0a4e2e0,8cff7e8c,d05c91e4,f5526eb4) at db_command+0x124
db_command_loop(d05c91e4,f5526e10,f5526e18,d040ca4d,800) at
db_command_loop+0x7
f
db_trap(1,0,58,d0,1) at db_trap+0xd0
kdb_trap(1,0,f5526eb4,f5526e7c,d05da7ef) at kdb_trap+0x10b
trap() at trap+0x3d7
ddb{3}> boot reboot
panic: tsleep: not SONPROC
...
OpenBSD 5.3-current (GENERIC.MP) #164: Sun Jun 2 17:36:48 MDT 2013
dera...@i386.openbsd.org:/usr/src/sys/arch/i386/compile/GENERIC.MP
cpu0: Intel(R) Xeon(R) CPU E5640 @ 2.67GHz ("GenuineIntel" 686-class)
2.67 GHz
cpu0:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,NXE,LONG,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,LAHF,PERF,ITSC
real mem = 3210981376 (3062MB)
avail mem = 3147075584 (3001MB)
mainbus0 at root
bios0 at mainbus0: AT/286+ BIOS, date 01/19/12, BIOS32 rev. 0 @ 0xfdb60,
SMBIOS rev. 2.5 @ 0xbf6c8000 (94 entries)
bios0: vendor FUJITSU // Phoenix Technologies Ltd. version "6.00 Rev.
1.13.2619.N1" date 01/19/2012
bios0: FUJITSU PRIMERGY RX300 S6
acpi0 at bios0: rev 2
acpi0: sleep states S0 S1 S4 S5
acpi0: tables DSDT FACP TCPA SLIT SSDT SSDT SSDT SSDT SSDT SSDT SSDT
SSDT EINJ HEST BERT SSDT ERST SPCR DMAR MCFG HPET APIC BOOT
acpi0: wakeup devices PE0_(S4) PE1_(S4) PE3_(S4) PE5_(S4) PE7_(S4)
PE8_(S4) PE9_(S4) PE10(S4) PE1A(S4) PE1E(S4) USB1(S1) USB2(S1) USB3(S1)
USB4(S1) USB5(S1) USB6(S1) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimcfg0 at acpi0 addr 0xe0000000, bus 0-10
acpihpet0 at acpi0: 14318179 Hz
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: apic clock running at 133MHz
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Xeon(R) CPU E5640 @ 2.67GHz ("GenuineIntel" 686-class)
2.67 GHz
cpu1:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,NXE,LONG,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,LAHF,PERF,ITSC
cpu2 at mainbus0: apid 18 (application processor)
cpu2: Intel(R) Xeon(R) CPU E5640 @ 2.67GHz ("GenuineIntel" 686-class)
2.67 GHz
cpu2:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,NXE,LONG,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,LAHF,PERF,ITSC
cpu3 at mainbus0: apid 20 (application processor)
cpu3: Intel(R) Xeon(R) CPU E5640 @ 2.67GHz ("GenuineIntel" 686-class)
2.67 GHz
cpu3:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,NXE,LONG,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AES,LAHF,PERF,ITSC
ioapic0 at mainbus0: apid 1 pa 0xfec00000, version 20, 24 pins
ioapic1 at mainbus0: apid 3 pa 0xfec80000, version 20, 24 pins
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 1 (PE1_)
acpiprt2 at acpi0: bus 2 (PE3_)
acpiprt3 at acpi0: bus 3 (PE5_)
acpiprt4 at acpi0: bus 4 (PE7_)
acpiprt5 at acpi0: bus 5 (PE8_)
acpiprt6 at acpi0: bus 6 (PE9_)
acpiprt7 at acpi0: bus 7 (PE10)
acpiprt8 at acpi0: bus 8 (PE1A)
acpiprt9 at acpi0: bus 9 (PE1E)
acpicpu0 at acpi0: C3, C2, C1, PSS
acpicpu1 at acpi0: C3, C2, C1, PSS
acpicpu2 at acpi0: C3, C2, C1, PSS
acpicpu3 at acpi0: C3, C2, C1, PSS
acpibtn0 at acpi0: PWRB
acpiac0 at acpi0: AC unit online
bios0: ROM list: 0xc0000/0x8000 0xc8000/0x2a00! 0xcb000/0x5c00
0xd0c00/0x200! 0xd0e00/0x1000 0xd1e00/0x1000
ipmi at mainbus0 not configured
cpu0: Enhanced SpeedStep 2667 MHz: speeds: 2661, 2660, 2527, 2394, 2261,
2128, 1995, 1862, 1729, 1596 MHz
pci0 at mainbus0 bus 0: configuration mode 1 (bios)
pchb0 at pci0 dev 0 function 0 "Intel 5520 Host" rev 0x13
ppb0 at pci0 dev 1 function 0 "Intel X58 PCIE" rev 0x13: apic 3 int 4
pci1 at ppb0 bus 1
mfi0 at pci1 dev 0 function 0 "Symbios Logic MegaRAID SAS2108 GEN2" rev
0x04: apic 3 int 4
mfi0: "RAID Ctrl SAS 6G 5/6 512MB (D2616)", firmware 12.4.0-0031, 512MB
cache
scsibus0 at mfi0: 64 targets
sd0 at scsibus0 targ 0 lun 0: <LSI, RAID 5/6 SAS 6G, 2.40> SCSI3
0/direct fixed naa.6003005700d24c20151664951f3f94ab
sd0: 139488MB, 512 bytes/sector, 285671424 sectors
ppb1 at pci0 dev 3 function 0 "Intel X58 PCIE" rev 0x13: apic 3 int 0
pci2 at ppb1 bus 2
em0 at pci2 dev 0 function 0 "Intel PRO/1000 PT (82571EB)" rev 0x06:
apic 3 int 0, address 00:1b:21:8c:41:96
em1 at pci2 dev 0 function 1 "Intel PRO/1000 PT (82571EB)" rev 0x06:
apic 3 int 10, address 00:1b:21:8c:41:97
ppb2 at pci0 dev 5 function 0 "Intel X58 PCIE" rev 0x13: apic 3 int 2
pci3 at ppb2 bus 3
ppb3 at pci0 dev 7 function 0 "Intel X58 PCIE" rev 0x13: apic 3 int 6
pci4 at ppb3 bus 4
ppb4 at pci0 dev 8 function 0 "Intel X58 PCIE" rev 0x13: apic 3 int 7
pci5 at ppb4 bus 5
ppb5 at pci0 dev 9 function 0 "Intel X58 PCIE" rev 0x13: apic 3 int 8
pci6 at ppb5 bus 6
ppb6 at pci0 dev 10 function 0 "Intel X58 PCIE" rev 0x13: apic 3 int 9
pci7 at ppb6 bus 7
"Intel X58 QuickPath" rev 0x13 at pci0 dev 16 function 0 not configured
"Intel X58 QuickPath" rev 0x13 at pci0 dev 16 function 1 not configured
"Intel X58 QuickPath" rev 0x13 at pci0 dev 17 function 0 not configured
"Intel X58 QuickPath" rev 0x13 at pci0 dev 17 function 1 not configured
"Intel X58 Misc" rev 0x13 at pci0 dev 20 function 0 not configured
"Intel X58 GPIO" rev 0x13 at pci0 dev 20 function 1 not configured
"Intel X58 RAS" rev 0x13 at pci0 dev 20 function 2 not configured
"Intel X58 Throttle" rev 0x13 at pci0 dev 20 function 3 not configured
vendor "Intel", unknown product 0x342f (class system subclass interrupt,
rev 0x13) at pci0 dev 21 function 0 not configured
uhci0 at pci0 dev 26 function 0 "Intel 82801JI USB" rev 0x00: apic 1 int 19
uhci1 at pci0 dev 26 function 1 "Intel 82801JI USB" rev 0x00: apic 1 int 18
uhci2 at pci0 dev 26 function 2 "Intel 82801JI USB" rev 0x00: apic 1 int 18
ehci0 at pci0 dev 26 function 7 "Intel 82801JI USB" rev 0x00: apic 1 int 18
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 "Intel EHCI root hub" rev 2.00/1.00 addr 1
ppb7 at pci0 dev 28 function 0 "Intel 82801JI PCIE" rev 0x00: apic 1 int 16
pci8 at ppb7 bus 8
em2 at pci8 dev 0 function 0 "Intel PRO/1000 PT (82575EB)" rev 0x02:
msi, address 00:19:99:98:e4:ea
em3 at pci8 dev 0 function 1 "Intel PRO/1000 PT (82575EB)" rev 0x02:
msi, address 00:19:99:98:e4:eb
ppb8 at pci0 dev 28 function 4 "Intel 82801JI PCIE" rev 0x00: apic 1 int 16
pci9 at ppb8 bus 9
vga1 at pci9 dev 0 function 0 "Matrox MGA G200e (ServerEngines)" rev 0x02
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
uhci3 at pci0 dev 29 function 0 "Intel 82801JI USB" rev 0x00: apic 1 int 23
uhci4 at pci0 dev 29 function 1 "Intel 82801JI USB" rev 0x00: apic 1 int 22
uhci5 at pci0 dev 29 function 2 "Intel 82801JI USB" rev 0x00: apic 1 int 21
ehci1 at pci0 dev 29 function 7 "Intel 82801JI USB" rev 0x00: apic 1 int 23
usb1 at ehci1: USB revision 2.0
uhub1 at usb1 "Intel EHCI root hub" rev 2.00/1.00 addr 1
ppb9 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev 0x90
pci10 at ppb9 bus 10
pcib0 at pci0 dev 31 function 0 "Intel 82801JIB LPC" rev 0x00
ahci0 at pci0 dev 31 function 2 "Intel 82801JI AHCI" rev 0x00: msi, AHCI 1.2
scsibus1 at ahci0: 32 targets
cd0 at scsibus1 targ 0 lun 0: <HL-DT-ST, DVDRAM_GT30N, FT03> ATAPI
5/cdrom removable
ichiic0 at pci0 dev 31 function 3 "Intel 82801JI SMBus" rev 0x00: apic 1
int 17
iic0 at ichiic0
iic0: addr 0x2e 00=40 words 00=4040 01=0000 02=0000 03=0000 04=0000
05=0000 06=0000 07=0000
usb2 at uhci0: USB revision 1.0
uhub2 at usb2 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb3 at uhci1: USB revision 1.0
uhub3 at usb3 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb4 at uhci2: USB revision 1.0
uhub4 at usb4 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb5 at uhci3: USB revision 1.0
uhub5 at usb5 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb6 at uhci4: USB revision 1.0
uhub6 at usb6 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb7 at uhci5: USB revision 1.0
uhub7 at usb7 "Intel UHCI root hub" rev 1.00/1.00 addr 1
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com0: console
com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5
kbc: cmd word write error
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
npx0 at isa0 port 0xf0/16: reported by CPUID; using exception 16
pci11 at mainbus0 bus 255
pchb1 at pci11 dev 0 function 0 vendor "Intel", unknown product 0x2c70
rev 0x02
pchb2 at pci11 dev 0 function 1 vendor "Intel", unknown product 0x2d81
rev 0x02
pchb3 at pci11 dev 2 function 0 vendor "Intel", unknown product 0x2d90
rev 0x02
pchb4 at pci11 dev 2 function 1 vendor "Intel", unknown product 0x2d91
rev 0x02
pchb5 at pci11 dev 2 function 2 vendor "Intel", unknown product 0x2d92
rev 0x02
pchb6 at pci11 dev 2 function 3 vendor "Intel", unknown product 0x2d93
rev 0x02
pchb7 at pci11 dev 2 function 4 vendor "Intel", unknown product 0x2d94
rev 0x02
pchb8 at pci11 dev 2 function 5 vendor "Intel", unknown product 0x2d95
rev 0x02
pchb9 at pci11 dev 3 function 0 vendor "Intel", unknown product 0x2d98
rev 0x02
pchb10 at pci11 dev 3 function 1 vendor "Intel", unknown product 0x2d99
rev 0x02
pchb11 at pci11 dev 3 function 2 vendor "Intel", unknown product 0x2d9a
rev 0x02
pchb12 at pci11 dev 3 function 4 vendor "Intel", unknown product 0x2d9c
rev 0x02
pchb13 at pci11 dev 4 function 0 vendor "Intel", unknown product 0x2da0
rev 0x02
pchb14 at pci11 dev 4 function 1 vendor "Intel", unknown product 0x2da1
rev 0x02
pchb15 at pci11 dev 4 function 2 vendor "Intel", unknown product 0x2da2
rev 0x02
pchb16 at pci11 dev 4 function 3 vendor "Intel", unknown product 0x2da3
rev 0x02
pchb17 at pci11 dev 5 function 0 vendor "Intel", unknown product 0x2da8
rev 0x02
pchb18 at pci11 dev 5 function 1 vendor "Intel", unknown product 0x2da9
rev 0x02
pchb19 at pci11 dev 5 function 2 vendor "Intel", unknown product 0x2daa
rev 0x02
pchb20 at pci11 dev 5 function 3 vendor "Intel", unknown product 0x2dab
rev 0x02
pchb21 at pci11 dev 6 function 0 vendor "Intel", unknown product 0x2db0
rev 0x02
pchb22 at pci11 dev 6 function 1 vendor "Intel", unknown product 0x2db1
rev 0x02
pchb23 at pci11 dev 6 function 2 vendor "Intel", unknown product 0x2db2
rev 0x02
pchb24 at pci11 dev 6 function 3 vendor "Intel", unknown product 0x2db3
rev 0x02
pci12 at mainbus0 bus 254
mtrr: Pentium Pro MTRR support
uhidev0 at uhub7 port 1 configuration 1 interface 0 "SILITEK USB
Keyboard and Mouse" rev 1.00/1.20 addr 2
uhidev0: iclass 3/1
ukbd0 at uhidev0: 8 variable keys, 6 key codes
wskbd0 at ukbd0 mux 1
wskbd0: connecting to wsdisplay0
uhidev1 at uhub7 port 1 configuration 1 interface 1 "SILITEK USB
Keyboard and Mouse" rev 1.00/1.20 addr 2
uhidev1: iclass 3/0, 2 report ids
ums0 at uhidev1 reportid 1: 3 buttons, Z dir
wsmouse0 at ums0 mux 0
uhid0 at uhidev1 reportid 2: input=1, output=0, feature=0
vscsi0 at root
scsibus2 at vscsi0: 256 targets
softraid0 at root
scsibus3 at softraid0: 256 targets
root on sd0a (45a94a78df33ffd9.a) swap on sd0b dump on sd0b
WARNING: / was not properly unmounted
carp: carp0 demoted group carp by 1 to 129 (carpdev)
carp: carp1 demoted group carp by 1 to 130 (carpdev)
carp: carp2 demoted group carp by 1 to 131 (carpdev)
carp: carp3 demoted group carp by 1 to 132 (carpdev)
carp: pfsync0 demoted group carp by 32 to 164 (pfsync init)
carp: pfsync0 demoted group pfsync by 32 to 32 (pfsync init)
carp: pfsync0 demoted group carp by 1 to 165 (pfsync bulk start)
carp: pfsync0 demoted group pfsync by 1 to 33 (pfsync bulk start)
carp: carp1 demoted group carp by -1 to 164 (carpdev)
carp: carp0 demoted group carp by -1 to 163 (carpdev)
carp: carp3 demoted group carp by -1 to 162 (carpdev)
carp: carp2 demoted group carp by -1 to 161 (carpdev)
carp1: state transition: BACKUP -> MASTER
arp_rtrequest: bad gateway value
carp0: state transition: BACKUP -> MASTER
arp_rtrequest: bad gateway value
carp3: state transition: BACKUP -> MASTER
arp_rtrequest: bad gateway value
carp2: state transition: BACKUP -> MASTER
arp_rtrequest: bad gateway value
carp0: state transition: MASTER -> BACKUP
carp3: state transition: MASTER -> BACKUP
carp2: state transition: MASTER -> BACKUP
carp1: state transition: MASTER -> BACKUP
carp: pfsync0 demoted group carp by -1 to 32 (pfsync bulk done)
carp: pfsync0 demoted group pfsync by -1 to 32 (pfsync bulk done)
carp: pfsync0 demoted group carp by -32 to 0 (pfsync init)
carp: pfsync0 demoted group pfsync by -32 to 0 (pfsync init)
carp0: state transition: BACKUP -> MASTER
arp_rtrequest: bad gateway value
carp3: state transition: BACKUP -> MASTER
arp_rtrequest: bad gateway value
carp2: state transition: BACKUP -> MASTER
arp_rtrequest: bad gateway value
carp1: state transition: BACKUP -> MASTER
arp_rtrequest: bad gateway value
