Re: carp backup becomes no carrier

2015-04-28 Thread Johan Huldtgren

hello,


I doubt the problem lies in your setup, I found the master->active
problem, diff below should correct that, can you tell me if it helps?

Index: netinet/ip_carp.c
===
RCS file: /cvs/src/sys/netinet/ip_carp.c,v
retrieving revision 1.253
diff -u -p -r1.253 ip_carp.c
--- netinet/ip_carp.c   22 Apr 2015 06:44:17 -  1.253
+++ netinet/ip_carp.c   28 Apr 2015 09:31:07 -
@@ -750,6 +750,7 @@ carp_clone_create(ifc, unit)
if_attach(ifp);
ether_ifattach(ifp);
ifp->if_type = IFT_CARP;
+   ifp->if_sadl->sdl_type = IFT_CARP;
ifp->if_output = carp_output;

/* Hook carp_addr_updated to cope with address and route changes. */


ok, with this patch status is once again "master" and "backup", overall 
behavior
also seems better, I'm not seeing periods when I can't reach the carp 
interfaces
after a failover. I believe this fixes my issues, I've been testing all 
morning
with both reboot failovers and increasing the demote counters, all seems 
well

now.

thanks,

.jh




Re: carp backup becomes no carrier

2015-04-28 Thread Peter Hessler
On 2015 Apr 28 (Tue) at 11:36:00 +0200 (+0200), Martin Pieuchot wrote:
:Hello Johan and thanks for your great report!
:
:On 27/04/15(Mon) 11:54, Johan Huldtgren wrote:
:> >If you try 1.250 and 1.253 and tell me if you can reproduce the problem
:> >that would be really helpful.  In case you see something weird, Could
:> >you include the routing table "netstat -rnf inet" in your report?  If
:> >you can also play with tcpdump on the various pseudo-interfaces and see
:> >if something is wrong that would be great.
:> 
:> i'm not sure what is going on, it seems likely that the problem lies
:> in my setup or something I've done, I'll explain the behavior I'm
:> seeing and hopefully you can use it to rule out that the error lies
:> in ip_carp.c at least.
:
:I doubt the problem lies in your setup, I found the master->active
:problem, diff below should correct that, can you tell me if it helps?
:
:Index: netinet/ip_carp.c
:===
:RCS file: /cvs/src/sys/netinet/ip_carp.c,v
:retrieving revision 1.253
:diff -u -p -r1.253 ip_carp.c
:--- netinet/ip_carp.c  22 Apr 2015 06:44:17 -  1.253
:+++ netinet/ip_carp.c  28 Apr 2015 09:31:07 -
:@@ -750,6 +750,7 @@ carp_clone_create(ifc, unit)
:   if_attach(ifp);
:   ether_ifattach(ifp);
:   ifp->if_type = IFT_CARP;
:+  ifp->if_sadl->sdl_type = IFT_CARP;
:   ifp->if_output = carp_output;
: 
:   /* Hook carp_addr_updated to cope with address and route changes. */
:

Untested, but that is certainly a problem

OK

-- 
A billion here, a couple of billion there -- first thing you know it
adds up to be real money.
-- Senator Everett McKinley Dirksen



Re: carp backup becomes no carrier

2015-04-28 Thread Martin Pieuchot
Hello Johan and thanks for your great report!

On 27/04/15(Mon) 11:54, Johan Huldtgren wrote:
> >If you try 1.250 and 1.253 and tell me if you can reproduce the problem
> >that would be really helpful.  In case you see something weird, Could
> >you include the routing table "netstat -rnf inet" in your report?  If
> >you can also play with tcpdump on the various pseudo-interfaces and see
> >if something is wrong that would be great.
> 
> i'm not sure what is going on, it seems likely that the problem lies
> in my setup or something I've done, I'll explain the behavior I'm
> seeing and hopefully you can use it to rule out that the error lies
> in ip_carp.c at least.

I doubt the problem lies in your setup, I found the master->active
problem, diff below should correct that, can you tell me if it helps?

Index: netinet/ip_carp.c
===
RCS file: /cvs/src/sys/netinet/ip_carp.c,v
retrieving revision 1.253
diff -u -p -r1.253 ip_carp.c
--- netinet/ip_carp.c   22 Apr 2015 06:44:17 -  1.253
+++ netinet/ip_carp.c   28 Apr 2015 09:31:07 -
@@ -750,6 +750,7 @@ carp_clone_create(ifc, unit)
if_attach(ifp);
ether_ifattach(ifp);
ifp->if_type = IFT_CARP;
+   ifp->if_sadl->sdl_type = IFT_CARP;
ifp->if_output = carp_output;
 
/* Hook carp_addr_updated to cope with address and route changes. */



Re: carp backup becomes no carrier

2015-04-27 Thread Johan Huldtgren

hello,


If you try 1.250 and 1.253 and tell me if you can reproduce the problem
that would be really helpful.  In case you see something weird, Could
you include the routing table "netstat -rnf inet" in your report?  If
you can also play with tcpdump on the various pseudo-interfaces and see
if something is wrong that would be great.


i'm not sure what is going on, it seems likely that the problem lies
in my setup or something I've done, I'll explain the behavior I'm
seeing and hopefully you can use it to rule out that the error lies
in ip_carp.c at least.

As I stated originally I have two carp nodes, one is still running
the April 12th snap, the other is running the April 23rd snap but with
a recompiled kernel containing ip_carp.c r=1.249, 1,250, or 1.253. The
host running the April 12th snap is normally the backup, it has an
advskew of 100 set in /etc/hostname.carp*

If the April 23rd host is master and a failover occurs, I can not
reach the carp interfaces on the April 12th node for several minutes,
unless the April 23rd host is shutdown / rebooted. If the failover
goes the other way from the April 12th host to the April 23rd host,
this does not happen, here I never lose a ping. This behavior is seen
regardless of which version of ip_carp.c I'm using.

Further when a reboot is initiated on the April 23rd host and the
April 12th host becomes the master, it stays the master even when
the April 23rd host returns and takes over (resulting in two masters)
At this point 'ifconfig -g carp [-]carpdemote 120' is only sometimes
successful in forcing one of the nodes into backup mode. With April
23rd running with version 1.253 I never managed this, had to take a
reboot on the April 12th host to make one of them end up the backup.
I am assuming this has to do with this error:

pfsync: failed to receive bulk update

I will note though that my initial report regarding carp interfaces
showing up as "no carrier" happens on all kernels with ip_carp.c
after r=1.249 While I missed it originally, it seems that also the
status when a carp interface is master is changed and is now
reported as "active" as opposed to "master", it's almost like they
are inheriting their status from what one would expect the parent
interface to have.

Below is ifconfig and netstat output for each version both as master
and backup. tcpdump on the carp interfaces didn't reveal anything to
me that looked out of the ordinary, but if there is something specific
you'd like to look for or just include output from them when things
aren't working I can do that.

thanks,

.jh


###
ip_carp.c r=1.249
###

$ ifconfig
lo0: flags=8049 mtu 32768
priority: 0
groups: lo
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x7
inet6 ::1 prefixlen 128
inet 127.0.0.1 netmask 0xff00
vr0: flags=8b43 
mtu 1500

lladdr 00:00:24:c8:da:54
priority: 0
groups: egress
media: Ethernet autoselect (100baseTX full-duplex)
status: active
inet 172.16.0.3 netmask 0xff00 broadcast 172.16.0.255
vr1: flags=8b43 
mtu 1500

lladdr 00:00:24:c8:da:55
priority: 0
media: Ethernet autoselect (100baseTX full-duplex)
status: active
vr2: flags=8b43 
mtu 1500

lladdr 00:00:24:c8:da:56
priority: 0
media: Ethernet autoselect (100baseTX full-duplex)
status: active
vr3: flags=8b43 
mtu 1500

lladdr 00:00:24:c8:da:57
priority: 0
media: Ethernet autoselect (100baseTX full-duplex)
status: active
ipw0: flags=8802 mtu 1500
lladdr 00:04:23:83:7a:a1
priority: 4
groups: wlan
media: IEEE802.11 autoselect (autoselect mode 11b)
status: no network
ieee80211: nwid "" 100dBm
enc0: flags=0<>
priority: 0
groups: enc
status: active
vlan20: flags=8943 mtu 
1500

lladdr 00:00:24:c8:da:55
priority: 0
vlan: 20 parent interface: vr1
groups: vlan
status: active
inet 192.168.100.3 netmask 0xff00 broadcast 192.168.100.255
vlan30: flags=8943 mtu 
1500

lladdr 00:00:24:c8:da:56
priority: 0
vlan: 30 parent interface: vr2
groups: vlan
status: active
inet 192.168.0.3 netmask 0xff00 broadcast 192.168.0.255
vlan666: flags=8943 mtu 
1500

lladdr 00:00:24:c8:da:57
priority: 0
vlan: 666 parent interface: vr3
groups: vlan
status: active
inet 10.66.66.3 netmask 0xff00 broadcast 10.66.66.255
tun0: flags=8051 mtu 1500
priority: 0
groups: tun
status: active
inet 10.6.6.1 --> 10.6.6.2 netmask 0x
pfsync0: flags=41 mtu 1500
priority: 0
pfsync: syncdev: vlan666 syncpeer: 10.66.66.2 maxupd: 128 defer: 
off

groups: carp pfsync
pflog0: flags=141 mtu 33192
priority: 0
groups: pflog
carp0: flags=8843 mtu 1500
lladdr 00:00:5e:00:01:01
priority: 0

Re: carp backup becomes no carrier

2015-04-27 Thread Martin Pieuchot
On 24/04/15(Fri) 21:21, Johan Huldtgren wrote:
> a few hours after I sent the previous e-mail the backup
> (April 23rd snap) took over and became the master, at
> that point I could not reach the carp interfaces anymore.
> Reverting roles so the host running the April 12th snap
> became the master would mostly fix the problems although
> occasionally things would seem to get confused and traffic
> (esp to vlan666, which my laptop isn't on but has access
> to) would cease. Shutting down the node running the April
> 23rd snap would generally clear this up, but I'm not sure
> if this is a red herring and there is some caching going
> on somewhere which is clouding my troubleshooting efforts.

This looks like the bug I introduced in r1.251 that should be
fixed in 1.252 and 1.253, I'm puzzled.

> Regardless, I stood up an i386 vm, downloaded -current but
> grabbed ip_carp.c r1.249, and built a new kernel. Copied it
> over to the firewall which had the April 23rd snap and now
> everything is working as it was before. Traffic is flowing
> as expected regardless of which host is master and which is
> backup. It's only been a few hours, but so far so good.

If you try 1.250 and 1.253 and tell me if you can reproduce the problem
that would be really helpful.  In case you see something weird, Could
you include the routing table "netstat -rnf inet" in your report?  If
you can also play with tcpdump on the various pseudo-interfaces and see
if something is wrong that would be great.

> >I noticed some carp weirdness and sthen@ thought it might be worth
> >bringing to light.  Quick background, I run two carp nodes, one
> >(current master) is running the April 12th snapshot, the other is
> >running the April 23rd snapshot. The node running the April 23rd
> >snap when it's the backup node ifconfig reports all the carp
> >interfaces status' as "no carrier" whereas before (as far as I can
> >remember and on the April 12th snap at least) it would report
> >"backup". Once the backup becomes the master status changes to
> >"master".
> >
> >I don't notice anything not working, however this behavior is perhaps
> >not expected.

It's indeed no expected and generally shown for interfaces of type
IFT_ETHER.  I don't understand how this can be shown for carp.

> >dmesgs and ifconfig output for each host below, let me know if you
> >need anything further.
> >
> >thanks,
> >
> >.jh
> >
> >April 12th snapshot host:
> >
> >$ dmesg
> >syncing disks... done
> >OpenBSD 5.7-current (GENERIC) #772: Sun Apr 12 17:38:03 MDT 2015
> >dera...@i386.openbsd.org:/usr/src/sys/arch/i386/compile/GENERIC
> >cpu0: Geode(TM) Integrated Processor by AMD PCS ("AuthenticAMD"
> >586-class) 500 MHz
> >cpu0: FPU,DE,PSE,TSC,MSR,CX8,SEP,PGE,CMOV,CFLUSH,MMX,MMXX,3DNOW2,3DNOW
> >real mem  = 536363008 (511MB)
> >avail mem = 515301376 (491MB)
> >mpath0 at root
> >scsibus0 at mpath0: 256 targets
> >mainbus0 at root
> >bios0 at mainbus0: date 20/71/05, BIOS32 rev. 0 @ 0xfac40
> >pcibios0 at bios0: rev 2.0 @ 0xf/0x1
> >pcibios0: pcibios_get_intr_routing - function not supported
> >pcibios0: PCI IRQ Routing information unavailable.
> >pcibios0: PCI bus #0 is the last bus
> >bios0: ROM list: 0xc8000/0xa800
> >cpu0 at mainbus0: (uniprocessor)
> >mtrr: K6-family MTRR support (2 registers)
> >amdmsr0 at mainbus0
> >pci0 at mainbus0 bus 0: configuration mode 1 (no bios)
> >0:20:0: io address conflict 0x6100/0x100
> >0:20:0: io address conflict 0x6200/0x200
> >pchb0 at pci0 dev 1 function 0 "AMD Geode LX" rev 0x31
> >glxsb0 at pci0 dev 1 function 2 "AMD Geode LX Crypto" rev 0x00: RNG AES
> >vr0 at pci0 dev 6 function 0 "VIA VT6105M RhineIII" rev 0x96: irq 11,
> >address 00:00:24:c9:58:4c
> >ukphy0 at vr0 phy 1: Generic IEEE 802.3u media interface, rev. 3: OUI
> >0x004063, model 0x0034
> >vr1 at pci0 dev 7 function 0 "VIA VT6105M RhineIII" rev 0x96: irq 5,
> >address 00:00:24:c9:58:4d
> >ukphy1 at vr1 phy 1: Generic IEEE 802.3u media interface, rev. 3: OUI
> >0x004063, model 0x0034
> >vr2 at pci0 dev 8 function 0 "VIA VT6105M RhineIII" rev 0x96: irq 9,
> >address 00:00:24:c9:58:4e
> >ukphy2 at vr2 phy 1: Generic IEEE 802.3u media interface, rev. 3: OUI
> >0x004063, model 0x0034
> >vr3 at pci0 dev 9 function 0 "VIA VT6105M RhineIII" rev 0x96: irq 12,
> >address 00:00:24:c9:58:4f
> >ukphy3 at vr3 phy 1: Generic IEEE 802.3u media interface, rev. 3: OUI
> >0x004063, model 0x0034
> >ral0 at pci0 dev 17 function 0 "Ralink RT2561S" rev 0x00: irq 15,
> >address 00:12:0e:61:7f:b0
> >ral0: MAC/BBP RT2561C, RF RT5225
> >glxpcib0 at pci0 dev 20 function 0 "AMD CS5536 ISA" rev 0x03: rev 3,
> >32-bit 3579545Hz timer, watchdog, gpio, i2c
> >gpio0 at glxpcib0: 32 pins
> >iic0 at glxpcib0
> >pciide0 at pci0 dev 20 function 2 "AMD CS5536 IDE" rev 0x01: DMA,
> >channel 0 wired to compatibility, channel 1 wired to compatibility
> >wd0 at pciide0 channel 0 drive 0: 
> >wd0: 4-sector PIO, LBA, 7815MB, 16007040 sectors
> >wd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 2
> >pciide0: cha

Re: carp backup becomes no carrier

2015-04-24 Thread Johan Huldtgren

hello,

a few hours after I sent the previous e-mail the backup
(April 23rd snap) took over and became the master, at
that point I could not reach the carp interfaces anymore.
Reverting roles so the host running the April 12th snap
became the master would mostly fix the problems although
occasionally things would seem to get confused and traffic
(esp to vlan666, which my laptop isn't on but has access
to) would cease. Shutting down the node running the April
23rd snap would generally clear this up, but I'm not sure
if this is a red herring and there is some caching going
on somewhere which is clouding my troubleshooting efforts.

Regardless, I stood up an i386 vm, downloaded -current but
grabbed ip_carp.c r1.249, and built a new kernel. Copied it
over to the firewall which had the April 23rd snap and now
everything is working as it was before. Traffic is flowing
as expected regardless of which host is master and which is
backup. It's only been a few hours, but so far so good.

thanks,

.jh


On 2015-04-24 13:15, Johan Huldtgren wrote:

hello,

I noticed some carp weirdness and sthen@ thought it might be worth
bringing to light.  Quick background, I run two carp nodes, one
(current master) is running the April 12th snapshot, the other is
running the April 23rd snapshot. The node running the April 23rd
snap when it's the backup node ifconfig reports all the carp
interfaces status' as "no carrier" whereas before (as far as I can
remember and on the April 12th snap at least) it would report
"backup". Once the backup becomes the master status changes to
"master".

I don't notice anything not working, however this behavior is perhaps
not expected.

dmesgs and ifconfig output for each host below, let me know if you
need anything further.

thanks,

.jh

April 12th snapshot host:

$ dmesg
syncing disks... done
OpenBSD 5.7-current (GENERIC) #772: Sun Apr 12 17:38:03 MDT 2015
dera...@i386.openbsd.org:/usr/src/sys/arch/i386/compile/GENERIC
cpu0: Geode(TM) Integrated Processor by AMD PCS ("AuthenticAMD"
586-class) 500 MHz
cpu0: FPU,DE,PSE,TSC,MSR,CX8,SEP,PGE,CMOV,CFLUSH,MMX,MMXX,3DNOW2,3DNOW
real mem  = 536363008 (511MB)
avail mem = 515301376 (491MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: date 20/71/05, BIOS32 rev. 0 @ 0xfac40
pcibios0 at bios0: rev 2.0 @ 0xf/0x1
pcibios0: pcibios_get_intr_routing - function not supported
pcibios0: PCI IRQ Routing information unavailable.
pcibios0: PCI bus #0 is the last bus
bios0: ROM list: 0xc8000/0xa800
cpu0 at mainbus0: (uniprocessor)
mtrr: K6-family MTRR support (2 registers)
amdmsr0 at mainbus0
pci0 at mainbus0 bus 0: configuration mode 1 (no bios)
0:20:0: io address conflict 0x6100/0x100
0:20:0: io address conflict 0x6200/0x200
pchb0 at pci0 dev 1 function 0 "AMD Geode LX" rev 0x31
glxsb0 at pci0 dev 1 function 2 "AMD Geode LX Crypto" rev 0x00: RNG AES
vr0 at pci0 dev 6 function 0 "VIA VT6105M RhineIII" rev 0x96: irq 11,
address 00:00:24:c9:58:4c
ukphy0 at vr0 phy 1: Generic IEEE 802.3u media interface, rev. 3: OUI
0x004063, model 0x0034
vr1 at pci0 dev 7 function 0 "VIA VT6105M RhineIII" rev 0x96: irq 5,
address 00:00:24:c9:58:4d
ukphy1 at vr1 phy 1: Generic IEEE 802.3u media interface, rev. 3: OUI
0x004063, model 0x0034
vr2 at pci0 dev 8 function 0 "VIA VT6105M RhineIII" rev 0x96: irq 9,
address 00:00:24:c9:58:4e
ukphy2 at vr2 phy 1: Generic IEEE 802.3u media interface, rev. 3: OUI
0x004063, model 0x0034
vr3 at pci0 dev 9 function 0 "VIA VT6105M RhineIII" rev 0x96: irq 12,
address 00:00:24:c9:58:4f
ukphy3 at vr3 phy 1: Generic IEEE 802.3u media interface, rev. 3: OUI
0x004063, model 0x0034
ral0 at pci0 dev 17 function 0 "Ralink RT2561S" rev 0x00: irq 15,
address 00:12:0e:61:7f:b0
ral0: MAC/BBP RT2561C, RF RT5225
glxpcib0 at pci0 dev 20 function 0 "AMD CS5536 ISA" rev 0x03: rev 3,
32-bit 3579545Hz timer, watchdog, gpio, i2c
gpio0 at glxpcib0: 32 pins
iic0 at glxpcib0
pciide0 at pci0 dev 20 function 2 "AMD CS5536 IDE" rev 0x01: DMA,
channel 0 wired to compatibility, channel 1 wired to compatibility
wd0 at pciide0 channel 0 drive 0: 
wd0: 4-sector PIO, LBA, 7815MB, 16007040 sectors
wd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 2
pciide0: channel 1 ignored (disabled)
ohci0 at pci0 dev 21 function 0 "AMD CS5536 USB" rev 0x02: irq 7,
version 1.0, legacy support
ehci0 at pci0 dev 21 function 1 "AMD CS5536 USB" rev 0x02: irq 7
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 "AMD EHCI root hub" rev 2.00/1.00 addr 1
isa0 at glxpcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com0: console
com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5
pckbc0: unable to establish interrupt for aux slot
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
nsclpcsio0 at isa0 port 0x2e/2: NSC PC87366 rev 10: GPIO VLM TMS
gpio1 at nsclpcsio0: 29 pins
npx0 at isa0 port 0xf0/16: reported by CPUID; using e

carp backup becomes no carrier

2015-04-24 Thread Johan Huldtgren

hello,

I noticed some carp weirdness and sthen@ thought it might be worth
bringing to light.  Quick background, I run two carp nodes, one
(current master) is running the April 12th snapshot, the other is
running the April 23rd snapshot. The node running the April 23rd
snap when it's the backup node ifconfig reports all the carp
interfaces status' as "no carrier" whereas before (as far as I can
remember and on the April 12th snap at least) it would report
"backup". Once the backup becomes the master status changes to
"master".

I don't notice anything not working, however this behavior is perhaps
not expected.

dmesgs and ifconfig output for each host below, let me know if you
need anything further.

thanks,

.jh

April 12th snapshot host:

$ dmesg
syncing disks... done
OpenBSD 5.7-current (GENERIC) #772: Sun Apr 12 17:38:03 MDT 2015
dera...@i386.openbsd.org:/usr/src/sys/arch/i386/compile/GENERIC
cpu0: Geode(TM) Integrated Processor by AMD PCS ("AuthenticAMD" 
586-class) 500 MHz

cpu0: FPU,DE,PSE,TSC,MSR,CX8,SEP,PGE,CMOV,CFLUSH,MMX,MMXX,3DNOW2,3DNOW
real mem  = 536363008 (511MB)
avail mem = 515301376 (491MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: date 20/71/05, BIOS32 rev. 0 @ 0xfac40
pcibios0 at bios0: rev 2.0 @ 0xf/0x1
pcibios0: pcibios_get_intr_routing - function not supported
pcibios0: PCI IRQ Routing information unavailable.
pcibios0: PCI bus #0 is the last bus
bios0: ROM list: 0xc8000/0xa800
cpu0 at mainbus0: (uniprocessor)
mtrr: K6-family MTRR support (2 registers)
amdmsr0 at mainbus0
pci0 at mainbus0 bus 0: configuration mode 1 (no bios)
0:20:0: io address conflict 0x6100/0x100
0:20:0: io address conflict 0x6200/0x200
pchb0 at pci0 dev 1 function 0 "AMD Geode LX" rev 0x31
glxsb0 at pci0 dev 1 function 2 "AMD Geode LX Crypto" rev 0x00: RNG AES
vr0 at pci0 dev 6 function 0 "VIA VT6105M RhineIII" rev 0x96: irq 11, 
address 00:00:24:c9:58:4c
ukphy0 at vr0 phy 1: Generic IEEE 802.3u media interface, rev. 3: OUI 
0x004063, model 0x0034
vr1 at pci0 dev 7 function 0 "VIA VT6105M RhineIII" rev 0x96: irq 5, 
address 00:00:24:c9:58:4d
ukphy1 at vr1 phy 1: Generic IEEE 802.3u media interface, rev. 3: OUI 
0x004063, model 0x0034
vr2 at pci0 dev 8 function 0 "VIA VT6105M RhineIII" rev 0x96: irq 9, 
address 00:00:24:c9:58:4e
ukphy2 at vr2 phy 1: Generic IEEE 802.3u media interface, rev. 3: OUI 
0x004063, model 0x0034
vr3 at pci0 dev 9 function 0 "VIA VT6105M RhineIII" rev 0x96: irq 12, 
address 00:00:24:c9:58:4f
ukphy3 at vr3 phy 1: Generic IEEE 802.3u media interface, rev. 3: OUI 
0x004063, model 0x0034
ral0 at pci0 dev 17 function 0 "Ralink RT2561S" rev 0x00: irq 15, 
address 00:12:0e:61:7f:b0

ral0: MAC/BBP RT2561C, RF RT5225
glxpcib0 at pci0 dev 20 function 0 "AMD CS5536 ISA" rev 0x03: rev 3, 
32-bit 3579545Hz timer, watchdog, gpio, i2c

gpio0 at glxpcib0: 32 pins
iic0 at glxpcib0
pciide0 at pci0 dev 20 function 2 "AMD CS5536 IDE" rev 0x01: DMA, 
channel 0 wired to compatibility, channel 1 wired to compatibility

wd0 at pciide0 channel 0 drive 0: 
wd0: 4-sector PIO, LBA, 7815MB, 16007040 sectors
wd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 2
pciide0: channel 1 ignored (disabled)
ohci0 at pci0 dev 21 function 0 "AMD CS5536 USB" rev 0x02: irq 7, 
version 1.0, legacy support

ehci0 at pci0 dev 21 function 1 "AMD CS5536 USB" rev 0x02: irq 7
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 "AMD EHCI root hub" rev 2.00/1.00 addr 1
isa0 at glxpcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
com0: console
com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5
pckbc0: unable to establish interrupt for aux slot
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
nsclpcsio0 at isa0 port 0x2e/2: NSC PC87366 rev 10: GPIO VLM TMS
gpio1 at nsclpcsio0: 29 pins
npx0 at isa0 port 0xf0/16: reported by CPUID; using exception 16
usb1 at ohci0: USB revision 1.0
uhub1 at usb1 "AMD OHCI root hub" rev 1.00/1.00 addr 1
vscsi0 at root
scsibus1 at vscsi0: 256 targets
softraid0 at root
scsibus2 at softraid0: 256 targets
root on wd0a (ba730608caf94ae4.a) swap on wd0b dump on wd0b
carp0: state transition: BACKUP -> MASTER
carp1: state transition: BACKUP -> MASTER
carp2: state transition: BACKUP -> MASTER
carp3: state transition: BACKUP -> MASTER
carp0: state transition: MASTER -> BACKUP
carp1: state transition: MASTER -> BACKUP
carp2: state transition: MASTER -> BACKUP
carp3: state transition: MASTER -> BACKUP
carp3: state transition: BACKUP -> MASTER
carp2: state transition: BACKUP -> MASTER
carp1: state transition: BACKUP -> MASTER
carp0: state transition: BACKUP -> MASTER

$ ifconfig
lo0: flags=8049 mtu 32768
priority: 0
groups: lo
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x6
inet6 ::1 prefixlen 128
inet 127.0.0.1 netmask 0xff00
vr0: flags=8b43 
mtu 1500

lladdr 00:00:24:c9:58:4c
priority: 0
groups: egres