Hello,
OpenBSD has been my system of choice for router / firewall / access
point purposes since 2003 (v3.3). And naturally it's been doing great :)
Up until this year though, I would always use rather old i386 hardware
(15-20 years old PC's are still in operation), equipped with a bunch of
slow NIC's, while still dreaming about something neat, efficient, small,
silent and low-energy. Then, all of a sudden, a few months ago I
discovered Ubiquiti Networks' EdgeRouter PoE, followed by OpenBSD/octeon
port. Didn't need much time to decide to buy it, remove its original USB
flashdrive, deploy OpenBSD to a new one and finally give it a try. 6.0
Release is now installed, patched and configured with all the needed
packages. Below I include dmesg outputs ("sysctl hw.sensors" produces no
information):
Copyright (c) 1982, 1986, 1989, 1991, 1993
The Regents of the University of California. All rights reserved.
Copyright (c) 1995-2016 OpenBSD. All rights reserved.
http://www.OpenBSD.org
OpenBSD 6.0 (GENERIC) #10: Fri Jul 29 04:45:17 UTC 2016
visa@octeon:/usr/src/sys/arch/octeon/compile/GENERIC
real mem = 536870912 (512MB)
avail mem = 524386304 (500MB)
warning: no entropy supplied by boot loader
mainbus0 at root
cpu0 at mainbus0: Cavium OCTEON CPU rev 0.1 500 MHz, Software FP emulation
cpu0: cache L1-I 32KB 4 way D 8KB 64 way, L2 128KB 8 way
clock0 at mainbus0: int 5
iobus0 at mainbus0
dwctwo0 at iobus0 base 0x1180068000000 irq 56
usb0 at dwctwo0: USB revision 2.0
uhub0 at usb0 "Octeon DWC2 root hub" rev 2.00/1.00 addr 1
octrng0 at iobus0 base 0x1400000000000 irq 0
cn30xxgmx0 at iobus0 base 0x1180008000000
cnmac0 at cn30xxgmx0: RGMII, address **:**:**:**:**:**
atphy0 at cnmac0 phy 7: AR8035 10/100/1000 PHY, rev. 2
cnmac1 at cn30xxgmx0: RGMII, address **:**:**:**:**:**
atphy1 at cnmac1 phy 6: AR8035 10/100/1000 PHY, rev. 2
cnmac2 at cn30xxgmx0: RGMII, address **:**:**:**:**:**
uartbus0 at mainbus0
com0 at uartbus0 base 0x1180000000800 irq 34: ns16550a, 64 byte fifo
com0: console
com1 at uartbus0 base 0x1180000000c00 irq 35: ns16550a, 64 byte fifo
/dev/ksyms: Symbol table not valid.
umass0 at uhub0 port 1 configuration 1 interface 0 "JetFlash Mass
Storage Device" rev 2.10/11.00 addr 2
umass0: using SCSI over Bulk-Only
scsibus0 at umass0: 2 targets, initiator 0
sd0 at scsibus0 targ 1 lun 0: <JetFlash, Transcend 32GB, 1100> SCSI4
0/direct removable serial.********************
sd0: 30128MB, 512 bytes/sector, 61702144 sectors
vscsi0 at root
scsibus1 at vscsi0: 256 targets
softraid0 at root
scsibus2 at softraid0: 256 targets
boot device: sd0
root on sd0a (****************.a) swap on sd0b dump on sd0b
WARNING: No TOD clock, believing file system.
WARNING: CHECK AND RESET THE DATE!
Copyright (c) 1982, 1986, 1989, 1991, 1993
The Regents of the University of California. All rights reserved.
Copyright (c) 1995-2016 OpenBSD. All rights reserved.
http://www.OpenBSD.org
OpenBSD 6.0 (GENERIC.MP) #0: Thu Apr 6 00:45:05 CEST 2017
root@************************:/usr/src/sys/arch/octeon/compile/GENERIC.MP
real mem = 536870912 (512MB)
avail mem = 524353536 (500MB)
warning: no entropy supplied by boot loader
mainbus0 at root
cpu0 at mainbus0: Cavium OCTEON CPU rev 0.1 500 MHz, Software FP emulation
cpu0: cache L1-I 32KB 4 way D 8KB 64 way, L2 128KB 8 way
cpu1 at mainbus0: Cavium OCTEON CPU rev 0.1 500 MHz, Software FP emulation
cpu1: cache L1-I 32KB 4 way D 8KB 64 way, L2 128KB 8 way
clock0 at mainbus0: int 5
iobus0 at mainbus0
dwctwo0 at iobus0 base 0x1180068000000 irq 56
usb0 at dwctwo0: USB revision 2.0
uhub0 at usb0 "Octeon DWC2 root hub" rev 2.00/1.00 addr 1
octrng0 at iobus0 base 0x1400000000000 irq 0
cn30xxgmx0 at iobus0 base 0x1180008000000
cnmac0 at cn30xxgmx0: RGMII, address **:**:**:**:**:**
atphy0 at cnmac0 phy 7: AR8035 10/100/1000 PHY, rev. 2
cnmac1 at cn30xxgmx0: RGMII, address **:**:**:**:**:**
atphy1 at cnmac1 phy 6: AR8035 10/100/1000 PHY, rev. 2
cnmac2 at cn30xxgmx0: RGMII, address **:**:**:**:**:**
uartbus0 at mainbus0
com0 at uartbus0 base 0x1180000000800 irq 34: ns16550a, 64 byte fifo
com0: console
com1 at uartbus0 base 0x1180000000c00 irq 35: ns16550a, 64 byte fifo
/dev/ksyms: Symbol table not valid.
umass0 at uhub0 port 1 configuration 1 interface 0 "JetFlash Mass
Storage Device" rev 2.10/11.00 addr 2
umass0: using SCSI over Bulk-Only
scsibus0 at umass0: 2 targets, initiator 0
sd0 at scsibus0 targ 1 lun 0: <JetFlash, Transcend 32GB, 1100> SCSI4
0/direct removable serial.********************
sd0: 30128MB, 512 bytes/sector, 61702144 sectors
vscsi0 at root
scsibus1 at vscsi0: 256 targets
softraid0 at root
scsibus2 at softraid0: 256 targets
boot device: sd0
root on sd0a (****************.a) swap on sd0b dump on sd0b
WARNING: No TOD clock, believing file system.
WARNING: CHECK AND RESET THE DATE!
cpu1 launched
Copyright (c) 1982, 1986, 1989, 1991, 1993
The Regents of the University of California. All rights reserved.
Copyright (c) 1995-2017 OpenBSD. All rights reserved.
https://www.OpenBSD.org
OpenBSD 6.1-current (RAMDISK) #0: Mon Apr 17 09:56:55 UTC 2017
visa@octeon:/usr/src/sys/arch/octeon/compile/RAMDISK
real mem = 536870912 (512MB)
avail mem = 521076736 (496MB)
mainbus0 at root
cpu0 at mainbus0: CN50xx CPU rev 0.1 500 MHz, Software FP emulation
cpu0: cache L1-I 32KB 4 way D 8KB 64 way, L2 128KB 8 way
clock0 at mainbus0: int 5
iobus0 at mainbus0
dwctwo0 at iobus0 base 0x1180068000000 irq 56
usb0 at dwctwo0: USB revision 2.0
uhub0 at usb0 configuration 1 interface 0 "Octeon DWC2 root hub" rev
2.00/1.00 addr 1
cn30xxgmx0 at iobus0 base 0x1180008000000
cnmac0 at cn30xxgmx0: RGMII, address **:**:**:**:**:**
atphy0 at cnmac0 phy 7: AR8035 10/100/1000 PHY, rev. 2
cnmac1 at cn30xxgmx0: RGMII, address **:**:**:**:**:**
atphy1 at cnmac1 phy 6: AR8035 10/100/1000 PHY, rev. 2
cnmac2 at cn30xxgmx0: RGMII, address **:**:**:**:**:**
uartbus0 at mainbus0
com0 at uartbus0 base 0x1180000000800 irq 34: ns16550a, 64 byte fifo
com0: console
com1 at uartbus0 base 0x1180000000c00 irq 35: ns16550a, 64 byte fifo
umass0 at uhub0 port 1 configuration 1 interface 0 "JetFlash Mass
Storage Device" rev 2.10/11.00 addr 2
umass0: using SCSI over Bulk-Only
scsibus0 at umass0: 2 targets, initiator 0
sd0 at scsibus0 targ 1 lun 0: <JetFlash, Transcend 32GB, 1100> SCSI4
0/direct removable serial.********************
sd0: 30128MB, 512 bytes/sector, 61702144 sectors
root on rd0a swap on rd0b dump on rd0b
WARNING: No TOD clock, believing file system.
WARNING: CHECK AND RESET THE DATE!
***
The only, yet quite significant, problem I have is with cnmac2 NIC. As I
understand by checking its MAC address, this is how "switched" group of
ports (i.e. 3rd, 4th and 5th NIC) is seen by the kernel. And here are
the details of what's happening:
- looks like cnmac2 is missing its atphy* counterpart - atphy2:
"cnmac0 at cn30xxgmx0: RGMII, address **:**:**:**:**:**
atphy0 at cnmac0 phy 7: AR8035 10/100/1000 PHY, rev. 2
cnmac1 at cn30xxgmx0: RGMII, address **:**:**:**:**:**
atphy1 at cnmac1 phy 6: AR8035 10/100/1000 PHY, rev. 2
cnmac2 at cn30xxgmx0: RGMII, address **:**:**:**:**:**"
- "ifconfig cnmac2 media" shows rather strange output:
"cnmac2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
lladdr **:**:**:**:**:**
index 3 priority 0 llprio 3
media: Ethernet manual (none)
supported media:
media manual"
- despite setting IP address and the rest for cnmac2, no external
connections are working (both inbound and outbound) - I can ping this
address only from the router itself
According to Ubiquiti documentation, 3rd, 4th and 5th NIC's can either
be turned into a switch, or work separately. And having 5 independent
network interfaces of course would be something cool! However, for now I
cannot use these ports either way.
I did test them using the original operating system and they work fine.
Also, played with turning them into a switch and back - to check if that
would somehow affect how bsd.rd sees them after reboot - but it made no
difference.
***
And here is a couple of suggestions after my thorough reading of
"INSTALL.octeon" doc and experiments performed afterwards:
- replacing all occurrences of "rootdev=sd0" with "rootdev=/dev/sd0",
which fixes the issue with boot process stopping in the middle and
asking for "root device:"
- replacing all occurrences of "$loadaddr" with "${loadaddr}" - just a
cosmetic change
- putting an information that a kernel meant to be used, should actually
be copied to MSDOS partition, so that U-Boot can load it upon boot, e.g.
by doing this:
mount /dev/sd0i /mnt && cp /bsd /mnt/bsd && umount /mnt
***
Currently I use my new Ubiquiti access point with its original PoE
Injector. But certainly it would be great to be able to turn on 24V /
48V on chosen EdgeRouter network ports - like in the original operating
system. Although maybe this part of router software is closed source, so
it may not be that easy to implement in OpenBSD.
***
Another subject - crossing my fingers for successful implementation of
single user mode option when system starts. Unless it's already possible
somehow.
***
That's pretty much it about my current experience with OpenBSD/octeon on
EdgeRouter PoE. Let me know how I can help further.
And last but not least:
Many thanks to OpenBSD developers and other folks involved - for all the
good work across all these years. And best wishes on the 20th anniversary!
--
Cheers,
Pawel Waga
