Re: info about cpu in dmesg

[Hrvoje Popovski]

On 15.6.2024. 7:54, Rob Schmersel wrote:
> On Fri, 14 Jun 2024 22:20:55 +0200
> Hrvoje Popovski  wrote:
> 
>> Hi all,
>>
>> I have question about cpu output in dmesg.
>> I have Fujitsu RX2530m4 with 8 core Intel(R) Xeon(R) Gold 6134 and in
>> dmesg I've noticed that core are 0,4,5,7,18,19,21,22
>>
>> without HT
>> cpu0: smt 0, core 0, package 0
>> cpu1: smt 0, core 4, package 0
>> cpu2: smt 0, core 5, package 0
>> cpu3: smt 0, core 7, package 0
>> cpu4: smt 0, core 18, package 0
>> cpu5: smt 0, core 19, package 0
>> cpu6: smt 0, core 21, package 0
>> cpu7: smt 0, core 22, package 0
>>
>> with HT
>> cpu0: smt 0, core 0, package 0
>> cpu1: smt 0, core 4, package 0
>> cpu2: smt 0, core 5, package 0
>> cpu3: smt 0, core 7, package 0
>> cpu4: smt 0, core 18, package 0
>> cpu5: smt 0, core 19, package 0
>> cpu6: smt 0, core 21, package 0
>> cpu7: smt 0, core 22, package 0
>> cpu8: smt 1, core 0, package 0
>> cpu9: smt 1, core 4, package 0
>> cpu10: smt 1, core 5, package 0
>> cpu11: smt 1, core 7, package 0
>> cpu12: smt 1, core 18, package 0
>> cpu13: smt 1, core 19, package 0
>> cpu14: smt 1, core 21, package 0
>> cpu15: smt 1, core 22, package 0
>>
>> My understanding is that :
>> package  - cpu socket
>> core - physical cpu cores
>> smt  - core thread
>> cpuX - name of core ?
>>
>> I thought that in my case core should be from 0 to 7 ?
>>
> <--- snip dmesg --->
> 
> The Xeon gold 6000 series can have upto 22 cores. The 6134 just has
> some of those cores disabled
> 

Hi,

makes sense, tnx ..

info about cpu in dmesg

[Hrvoje Popovski]

Hi all,

I have question about cpu output in dmesg.
I have Fujitsu RX2530m4 with 8 core Intel(R) Xeon(R) Gold 6134 and in
dmesg I've noticed that core are 0,4,5,7,18,19,21,22

without HT
cpu0: smt 0, core 0, package 0
cpu1: smt 0, core 4, package 0
cpu2: smt 0, core 5, package 0
cpu3: smt 0, core 7, package 0
cpu4: smt 0, core 18, package 0
cpu5: smt 0, core 19, package 0
cpu6: smt 0, core 21, package 0
cpu7: smt 0, core 22, package 0

with HT
cpu0: smt 0, core 0, package 0
cpu1: smt 0, core 4, package 0
cpu2: smt 0, core 5, package 0
cpu3: smt 0, core 7, package 0
cpu4: smt 0, core 18, package 0
cpu5: smt 0, core 19, package 0
cpu6: smt 0, core 21, package 0
cpu7: smt 0, core 22, package 0
cpu8: smt 1, core 0, package 0
cpu9: smt 1, core 4, package 0
cpu10: smt 1, core 5, package 0
cpu11: smt 1, core 7, package 0
cpu12: smt 1, core 18, package 0
cpu13: smt 1, core 19, package 0
cpu14: smt 1, core 21, package 0
cpu15: smt 1, core 22, package 0

My understanding is that :
package - cpu socket
core- physical cpu cores
smt - core thread
cpuX- name of core ?

I thought that in my case core should be from 0 to 7 ?

Fujitsu RX2530m4 is two socket machine and up to 28 cores
https://sp.ts.fujitsu.com/dmsp/Publications/public/ds-py-rx2530-m4.pdf

I don't have problems with that machine, actually machine is great,
boots quite fast, lots of pcie slots and it worked in production for 5
years without any problems. I'm just puzzled why in dmesg I see "core"
that are higher that 7 ?


fw2# dmesg
OpenBSD 7.5-current (GENERIC.MP) #0: Fri Jun 14 17:46:43 CEST 2024
hrvoje@fw2.netlab:/sys/arch/amd64/compile/GENERIC.MP
real mem = 50646933504 (48300MB)
avail mem = 49088184320 (46814MB)
random: good seed from bootblocks
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 3.0 @ 0x6f93e000 (85 entries)
bios0: vendor FUJITSU // American Megatrends Inc. version "V5.0.0.12
R1.62.0 for D3383-A1x" date 07/24/2023
bios0: FUJITSU PRIMERGY RX2530 M4
acpi0 at bios0: ACPI 6.1
acpi0: sleep states S0 S5
acpi0: tables DSDT FACP FPDT FIDT SPMI UEFI UEFI MCEJ MCFG HPET APIC
MIGT MSCT PCAT PCCT RASF SLIT SRAT SVOS WDDT OEM4 OEM1 SSDT SSDT SSDT
DMAR HEST BERT ERST EINJ
acpi0: wakeup devices PWRB(S0) XHCI(S0) RP17(S0) PXSX(S0) RP18(S0)
PXSX(S0) RP19(S0) PXSX(S0) RP20(S0) PXSX(S0) RP01(S0) PXSX(S0) RP02(S0)
PXSX(S0) RP03(S0) PXSX(S0) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimcfg0 at acpi0
acpimcfg0: addr 0x8000, bus 0-255
acpihpet0 at acpi0: 2399 Hz
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(R) Gold 6134 CPU @ 3.20GHz, 3192.58 MHz, 06-55-04,
patch 02007006
cpu0: cpuid 1
edx=bfebfbff
ecx=77fefbff
cpu0: cpuid 6 eax=77 ecx=9
cpu0: cpuid 7.0
ebx=d39b
ecx=8 edx=bc002400
cpu0: cpuid a vers=4, gp=4, gpwidth=48, ff=3, ffwidth=48
cpu0: cpuid d.1 eax=f
cpu0: cpuid 8001 edx=2c100800
ecx=121
cpu0: cpuid 8007 edx=100
cpu0: msr 10a=2000c04
cpu0: MELTDOWN
cpu0: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 1MB
64b/line 16-way L2 cache, 24MB 64b/line 11-way L3 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 24MHz
cpu0: mwait min=64, max=64, C-substates=0.2.0.2, IBE
cpu1 at mainbus0: apid 8 (application processor)
cpu1: Intel(R) Xeon(R) Gold 6134 CPU @ 3.20GHz, 3192.70 MHz, 06-55-04,
patch 02007006
cpu1: smt 0, core 4, package 0
cpu2 at mainbus0: apid 10 (application processor)
cpu2: Intel(R) Xeon(R) Gold 6134 CPU @ 3.20GHz, 3192.74 MHz, 06-55-04,
patch 02007006
cpu2: smt 0, core 5, package 0
cpu3 at mainbus0: apid 14 (application processor)
cpu3: Intel(R) Xeon(R) Gold 6134 CPU @ 3.20GHz, 3192.79 MHz, 06-55-04,
patch 02007006
cpu3: smt 0, core 7, package 0
cpu4 at mainbus0: apid 36 (application processor)
cpu4: Intel(R) Xeon(R) Gold 6134 CPU @ 3.20GHz, 3193.21 MHz, 06-55-04,
patch 02007006
cpu4: smt 0, core 18, package 0
cpu5 at mainbus0: apid 38 (application processor)
cpu5: Intel(R) Xeon(R) Gold 6134 CPU @ 3.20GHz, 3193.32 MHz, 06-55-04,
patch 02007006
cpu5: smt 0, core 19, package 0
cpu6 at mainbus0: apid 42 (application processor)
cpu6: Intel(R) Xeon(R) Gold 6134 CPU @ 3.20GHz, 3192.68 MHz, 06-55-04,
patch 02007006
cpu6: smt 0, core 21, package 0
cpu7 at mainbus0: apid 44 (application processor)
cpu7: Intel(R) Xeon(R) Gold 6134 CPU @ 3.20GHz, 3193.07 MHz, 06-55-04,
patch 02007006
cpu7: smt 0, core 22, package 0
cpu8 at mainbus0: apid 1 (application processor)
cpu8: Intel(R) Xeon(R) Gold 6134 CPU @ 3.20GHz, 3194.59 MHz, 06-55-04,
patch 02007006
cpu8: smt 1, core 0, package 0
cpu9 at mainbus0: apid 9 (application processor)
cpu9: Intel(R) Xeon(R) Gold 6134 CPU @ 3.20GHz, 3194.57 MHz, 06-55-04,
patch 02007006
cpu9: smt 1, core 4, package 0
cpu10 at mainbus0: apid 11 (application processor)
cpu10: Intel(R) Xeon(R) Gold 6134 CPU @ 3.20GHz, 3194.68 MHz, 06-55-04,
patch 02007006
cpu10: smt 1, core 5, package 0
cpu11 at mainb

Re: Recommendations for 2.5G NIC

[Hrvoje Popovski]

On 6.6.2024. 6:08, s...@skolma.com wrote:
> 
> 
> On Thursday, June 6th, 2024 at 1:08 PM, Martin  wrote:
> 
>>
>>
>> I am about to upgrade a network from 1G to 2.5G and a couple
>> of boxes needs new NICs.
>>
> mee too.
> 
>> Any recommendations for NICs with good driver support on OpenBSD?
>>
>> It would be nice it the cards also run well on FreeBSD and Linux, if
>> you happen to know that, as a couple of boxes on the network run that
>> and I can perhaps stick to the same card, but it's not a requirement.
>>
> 
> i am also interested in others' answers.
> Just did a 'man -k "ethernet device" and looking through the list. some 2.5 
> called out, however some 10g ones run at multi gig.
> the intel 'igx' 225/226 adapters are available in pc/nucs/odroid etc.. but 
> assuming you wanted PCI addin cards intel will be the most cross platform 
> 'just works' choice.
> 
> i also bought 2 cablematters 2.5g usb3/c adapters to test out and seemed to 
> work. i think they used the realtek 'ure' driver.
> 
>> Thanks in advance.
>

Hi,

man igc

DESCRIPTION
 The igc driver supports Intel I225/I226 series Ethernet devices.

Serial console on vmware esxi 8

[Hrvoje Popovski]

Hi all,

this could be useful information to those who are using openbsd on vmware.

while testing jan@ vmx LRO diffs, openbsd machine panic. I've sent him
few screenshots and those screenshots are awful.
Good thing is that vmware have virtual serial port
https://docs.vmware.com/en/VMware-vSphere/8.0/vsphere-vm-administration/GUID-ACADB450-85A6-4F6E-9269-F11DC71D2564.html

I've used
1) - Right-click a virtual machine in the inventory and select Edit Settings
2) - On the Virtual Hardware tab, Add other device, expand Serial port,
and select a connection type
"Use named pipe"

On openbsd box
Serial port 1
use named pipe
pipe name - cons1
Near End - Server
Far End - A virtual machine

On linux box
Serial port 1
use named pipe
pipe name - cons1
Near End - Client
Far End - A virtual machine


On openbsd box you will see com0 in dmesg and
/etc/boot.conf
stty com0 115200
set tty com0

/etc/ttys
tty00   "/usr/libexec/getty std.115200" vt220   on  secure


On linux box install minicom and configure it for ttyS0 with default
connections settings


I'm not big vmware user, but this is cool :)

Re: [7.5/amd64] ipsec + npppd + sasyncd + carp - doesn't pick up the VPN session at switchover

[Hrvoje Popovski]

On 29.5.2024. 12:48, Radek wrote:
> Thank you, that explains everything. 
> Does wireguard support replication? Will it work properly in my CARP setup?
> 

Hi,

I have wg listen on carp interface for redundancy and it's working
without admins or clients needs to do anything when primary carp
firewall shuts down or even reboot. People will notice something
happened but wg vpn would start to work after cca 20 seconds.

root@pc-hrvoje:~# ping 10.2.0.1
PING 10.2.0.1 (10.2.0.1) 56(84) bytes of data.
64 bytes from 10.2.0.1: icmp_seq=1 ttl=254 time=1.46 ms
64 bytes from 10.2.0.1: icmp_seq=2 ttl=254 time=1.48 ms
64 bytes from 10.2.0.1: icmp_seq=3 ttl=254 time=2.24 ms
64 bytes from 10.2.0.1: icmp_seq=4 ttl=254 time=8.62 ms
64 bytes from 10.2.0.1: icmp_seq=5 ttl=254 time=1.33 ms
64 bytes from 10.2.0.1: icmp_seq=6 ttl=254 time=2.03 ms
64 bytes from 10.2.0.1: icmp_seq=7 ttl=254 time=5.79 ms
64 bytes from 10.2.0.1: icmp_seq=8 ttl=254 time=7.35 ms
64 bytes from 10.2.0.1: icmp_seq=9 ttl=254 time=2.05 ms
64 bytes from 10.2.0.1: icmp_seq=10 ttl=254 time=1.50 ms
64 bytes from 10.2.0.1: icmp_seq=11 ttl=254 time=2.34 ms
64 bytes from 10.2.0.1: icmp_seq=12 ttl=254 time=2.55 ms
64 bytes from 10.2.0.1: icmp_seq=28 ttl=254 time=7.69 ms
64 bytes from 10.2.0.1: icmp_seq=29 ttl=254 time=1.32 ms
64 bytes from 10.2.0.1: icmp_seq=30 ttl=254 time=3.37 ms
64 bytes from 10.2.0.1: icmp_seq=31 ttl=254 time=6.52 ms
64 bytes from 10.2.0.1: icmp_seq=32 ttl=254 time=11.0 ms
64 bytes from 10.2.0.1: icmp_seq=33 ttl=254 time=1.88 ms
^C


why not use iked as vpn solution ? i'm not sure but i think that iked is
working with sasyncd ...

Re: ixl driver very poor network performance

[Hrvoje Popovski]

On 16.4.2024. 20:22, Szél Gábor wrote:
> Dear @misc!
> 
> We have several more complex networks where openbsd is the router.
> 
> Structure of the network:
> 
>   * OpenBSD redundant routers
> - two OpenBSD
> - CARP
> - pfsync
> - LACP trunks for LAN (2x 10Gbit)  (1 side switch #1, 2 side switch
> #2 + VPC )  use OpenBSD aggr device
>   * Cisco Nexus 3K switch-es
> - VPC (2x40Gbit)
> - redundant LACP links (1 side switch #1, 2 side switch #2 + VPC )
>   * many VLANs
>   * PF default block all trafic, and allowed traffic only
>   * the servers connected usually 2x10Gbit LACP
> 
> *hardware:*
> 
>   * we updated this system in one place to OpenBSD 7.4
> hardware: Dell PE 640 (2x Xeon Gold 6134 CPU, 64Gb RAM, Intel X710
> network cards)
>   * we migrated the settings from the previous system (OpenBSD 7.0)
> the previous hardware was different! (2x Xeon E5-2650, 64Gb RAM,
> Intel X520 network cards)
> 
> *Problem:*
> 
> After upgrade with hardware change, we have very poor network performance!!
> Example: A simple veeam backup restore that goes through the openbsd
> router hangs the network completely (very big lag)
> In this case, the SSH connection on the router is have lag!
> But OpenBSD dont have high CPU usage.
> 
> If i make simple iperf speed test from OpenBSD to other server (all
> device have 10Gbit LACP link):
> 
> [ ID] Interval   Transfer Bitrate
> [  5]   0.00-1.00   sec   171 MBytes  1.44 Gbits/sec  
> [  5]   1.00-2.00   sec   313 MBytes  2.63 Gbits/sec  
> [  5]   2.00-3.00   sec   398 MBytes  3.34 Gbits/sec  
> [  5]   3.00-4.00   sec   384 MBytes  3.22 Gbits/sec  
> [  5]   4.00-5.00   sec   419 MBytes  3.51 Gbits/sec  
> [  5]   5.00-6.00   sec   376 MBytes  3.16 Gbits/sec  
> [  5]   6.00-7.00   sec   325 MBytes  2.73 Gbits/sec  
> [  5]   7.00-8.00   sec   337 MBytes  2.82 Gbits/sec  
> [  5]   8.00-9.00   sec   339 MBytes  2.85 Gbits/sec  
> [  5]   9.00-10.00  sec   332 MBytes  2.78 Gbits/sec  
> [  5]  10.00-10.19  sec  62.5 MBytes  2.75 Gbits/sec  
> 
> Between other devices, servers, etc ... , the speed is perfectly fine
> (stable 9-10 Gbits/sec)
> Only routed performace is very-very slow.
> 
> if I make a speed test between two OpenBSDs (master router, backup router)
> Better value but not perfect:
> 
> [ ID] Interval   Transfer Bitrate
> [  5]   0.00-1.00   sec   740 MBytes  6.20 Gbits/sec  
> [  5]   1.00-2.00   sec   781 MBytes  6.55 Gbits/sec  
> [  5]   2.00-3.00   sec   784 MBytes  6.58 Gbits/sec  
> [  5]   3.00-4.00   sec   783 MBytes  6.57 Gbits/sec  
> [  5]   4.00-5.00   sec   786 MBytes  6.59 Gbits/sec  
> [  5]   5.00-6.00   sec   796 MBytes  6.68 Gbits/sec  
> [  5]   6.00-7.00   sec   779 MBytes  6.54 Gbits/sec  
> [  5]   7.00-8.00   sec   774 MBytes  6.49 Gbits/sec  
> [  5]   8.00-9.00   sec   780 MBytes  6.55 Gbits/sec  
> [  5]   9.00-10.00  sec   786 MBytes  6.59 Gbits/sec  
> [  5]  10.00-10.00  sec   640 KBytes  10.2 Gbits/sec  
> - - - - - - - - - - - - - - - - - - - - - - - - -
> [ ID] Interval   Transfer Bitrate
> [  5]   0.00-10.00  sec  7.61 GBytes  6.54 Gbits/sec 
> receiver
> 
> PF have ~2000 rules, but
> If i disabled PF on tested OpenBSD router, nothing changes.
> 
> we've run out of ideas, what would be worth watching?


Hi,

can you upgrade firewalls to 7.5 and change ixl cards with ix or mcx if
you have ?
Do you maybe have ipsec tunnels on firewalls ? If you have can disable
them for test?

Re: Dell PERC H745

[Hrvoje Popovski]

On 29.3.2024. 9:06, Kapetanakis Giannis wrote:
> 
> On 28/03/2024 20:17, Stuart Henderson wrote:
>> On 2024-03-28, Hrvoje Popovski  wrote:
>>> On 28.3.2024. 11:01, Kapetanakis Giannis wrote:
>>>> I'm looking for a new server to replace our firewall/routing.
>>>>
>>>> Would like to ask if PERC H745 is supported.
>>>>
>>>> mfi(4) lists
>>>>    -   Dell PERC 5/e, PERC 5/i, PERC 6/e, PERC 6/i, PERC H310, PERC
>>>>    H700, PERC H800
>>>>
>>>> Is this ok?
>>>>
>>>> Trying bsd.rd on a newer server with H755, it was NOT detected.
>>>> On Linux it is shown as
>>>> 65:00.0 RAID bus controller: Broadcom / LSI MegaRAID 12GSAS/PCIe Secure 
>>>> SAS39xx
>>>> DeviceName: SL3 RAID
>>>> Subsystem: Dell PERC H755 Front
>>>>
>>>> That is on 7.4, didn't try current.
>>>>
>>>> However the BOSS-S1 adapter with 2 x M.2 sticks was detected
>>>>
>>>> How about HBA330 Mini and/or PERC H730P Mini ?
>>>>
>>>> About CPUs, I'm between Intel Xeon Gold 5315Y @ 3.20GHz vs AMD EPYC 72F3
>>>> Both are 8 cores, I will put 2 x cpus. Haven't tried EPYC at all but looks 
>>>> more performant.
>>>>
>>>> G
>>>>
>>> Hi,
>>>
>>> don't go with BOSS adapter or HBA330. I have both adapters in lab and
>>> they just don't work.
>>> I have working OpenBSD on PERC H740p, PERC H740P Mini, PERC H330 mini,
>>> PERC H310 Mini. I can't remember but I think that H730p should work.
>> also working: PERC H710 Mini, PERC H755 Front (both mfii)
> 
> Thanks Hrvoje and Stuart all for the valuable info.
> 
> There are so many adapters given/updated by Dell every year, maybe we
> should update the man pages to add the working ones?
> 
> My BOSS-S1 Modular adapter is detected both on 7.4 and current.


Yes, BOSS and HBA330 are detected and maybe you will be able to install
openbsd on them, but with BOSS adapter box will eventually panic and
with HBA330 will deadlock

Re: Dell PERC H745

[Hrvoje Popovski]

On 28.3.2024. 17:40, Hrvoje Popovski wrote:
> On 28.3.2024. 11:01, Kapetanakis Giannis wrote:
>> I'm looking for a new server to replace our firewall/routing.
>>
>> Would like to ask if PERC H745 is supported.
>>
>> mfi(4) lists
>>    -   Dell PERC 5/e, PERC 5/i, PERC 6/e, PERC 6/i, PERC H310, PERC
>>    H700, PERC H800
>>
>> Is this ok?
>>
>> Trying bsd.rd on a newer server with H755, it was NOT detected.
>> On Linux it is shown as
>> 65:00.0 RAID bus controller: Broadcom / LSI MegaRAID 12GSAS/PCIe Secure 
>> SAS39xx
>> DeviceName: SL3 RAID
>> Subsystem: Dell PERC H755 Front
>>
>> That is on 7.4, didn't try current.
>>
>> However the BOSS-S1 adapter with 2 x M.2 sticks was detected
>>
>> How about HBA330 Mini and/or PERC H730P Mini ?
>>
>> About CPUs, I'm between Intel Xeon Gold 5315Y @ 3.20GHz vs AMD EPYC 72F3
>> Both are 8 cores, I will put 2 x cpus. Haven't tried EPYC at all but looks 
>> more performant.
>>
>> G
>>
> 
> Hi,
> 
> don't go with BOSS adapter or HBA330. I have both adapters in lab and
> they just don't work.
> I have working OpenBSD on PERC H740p, PERC H740P Mini, PERC H330 mini,
> PERC H310 Mini. I can't remember but I think that H730p should work.

Found it

Dell R7515 with PERC H730P Mini
AMD EPYC 7702P 64-Core Processor

mfii0 at pci1 dev 0 function 0 "Symbios Logic MegaRAID SAS3108" rev
0x02: msi
mfii0: "PERC H730P Mini", firmware 25.5.9.0001, 2048MB cache
scsibus1 at mfii0: 64 targets
sd0 at scsibus1 targ 0 lun 0: 
naa.64cd98f0cbb4aa002673b23f20452446
sd0: 457344MB, 512 bytes/sector, 936640512 sectors
scsibus2 at mfii0: 256 targets

Re: Dell PERC H745

[Hrvoje Popovski]

On 28.3.2024. 11:01, Kapetanakis Giannis wrote:
> I'm looking for a new server to replace our firewall/routing.
> 
> Would like to ask if PERC H745 is supported.
> 
> mfi(4) lists
>    -   Dell PERC 5/e, PERC 5/i, PERC 6/e, PERC 6/i, PERC H310, PERC
>    H700, PERC H800
> 
> Is this ok?
> 
> Trying bsd.rd on a newer server with H755, it was NOT detected.
> On Linux it is shown as
> 65:00.0 RAID bus controller: Broadcom / LSI MegaRAID 12GSAS/PCIe Secure 
> SAS39xx
> DeviceName: SL3 RAID
> Subsystem: Dell PERC H755 Front
> 
> That is on 7.4, didn't try current.
> 
> However the BOSS-S1 adapter with 2 x M.2 sticks was detected
> 
> How about HBA330 Mini and/or PERC H730P Mini ?
> 
> About CPUs, I'm between Intel Xeon Gold 5315Y @ 3.20GHz vs AMD EPYC 72F3
> Both are 8 cores, I will put 2 x cpus. Haven't tried EPYC at all but looks 
> more performant.
> 
> G
> 

Hi,

don't go with BOSS adapter or HBA330. I have both adapters in lab and
they just don't work.
I have working OpenBSD on PERC H740p, PERC H740P Mini, PERC H330 mini,
PERC H310 Mini. I can't remember but I think that H730p should work.




PowerEdge R740xd with H740P
Intel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz

sd0 at scsibus3 targ 0 lun 0: 
naa.6d09466073e86a002d956fda091d67f4
sd0: 915200MB, 512 bytes/sector, 1874329600 sectors

rs1# bioctl sd0
Volume  Status   Size Device
mfii0 0 Online   959656755200 sd0 RAID1 WB
  0 Online   960197124096 1:0.0   noencl 
  1 Online   960197124096 1:1.0   noencl 



PowerEdge R630 with PERC H330 Mini
Intel(R) Xeon(R) CPU E5-2637 v3 @ 3.50GHz

sd0 at scsibus1 targ 0 lun 0: 
naa.614187704a1f37001ddf7ffc11e3e762
sd0: 285568MB, 512 bytes/sector, 584843264 sectors

alt-fw1# bioctl sd0
Volume  Status   Size Device
mfii0 0 Online   299439751168 sd0 RAID1 WT
  0 Online   3000 1:0.0   noencl 
  1 Online   3000 1:1.0   noencl 



PowerEdge R6515 with HBA330
AMD EPYC 7313P 16-Core Processor
HBA300 doesn't work but if you have NVMe extender then U2 NVMe disk can
be attached to that extender through HBA330 connectors

mpii0: Dell HBA330 Mini, firmware 16.0.11.0, MPI 2.5 <- not working


nvme0 at pci13 dev 0 function 0 vendor "SK hynix", unknown product
0x2839 rev 0x21: msix, NVMe 1.3
nvme0: Dell DC NVMe PE8010 RI U.2 960GB, firmware 1.3.0, serial
SJC2N4257I34R2Q19

U2 NVMe disk is connected though HBA330 connectors to NVMe extender



PowerEdge R6515 with PERC H740P Mini
AMD EPYC 7313P 16-Core Processor

sd0 at scsibus1 targ 0 lun 0: 
naa.6f4ee08004838b002a3466dba8a488b1
sd0: 457344MB, 512 bytes/sector, 936640512 sectors

alt-fw2# bioctl sd0
Volume  Status   Size Device
mfii0 0 Online   479559942144 sd0 RAID1 WB
  0 Online   480103981056 1:0.0   noencl 
  1 Online   480103981056 1:1.0   noencl 

For me this server is beast because cpu clock can go up to 3.7GHz



It seems that on OpenBSD AMD cpus can have higher clock than Intel cpus

AMD EPYC 7313P 16-Core Processor
Base Clock - 3.0GHz
Max. Boost Clock - 3.7GHz
hw.cpuspeed=3000
hw.sensors.cpu0.frequency0=37.00 Hz


AMD EPYC 7413 24-Core Processor
Base Clock - 2.65GHz
Max. Boost Clock - 3.6GHz
hw.cpuspeed=2650
hw.sensors.cpu1.frequency0=34.00 Hz


Intel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz
Base Clock - 2.10 GHz
Max. Boost Clock - 3.70 GHz
hw.cpuspeed=2793
hw.sensors.cpu0.frequency0=28.00 Hz


Intel(R) Xeon(R) Gold 6134 CPU @ 3.20GHz
Base Clock - 3.20 GHz
Max. Boost Clock - 3.70 GHz
hw.cpuspeed=3201
hw.sensors.cpu0.frequency0=37.00 Hz
but this is fujitsu server :)


Intel(R) Xeon(R) CPU E5-2643 v2 @ 3.50GHz
Base Clock - 3.50 GHz
Max. Boost Clock - 3.80 GHz
hw.cpuspeed=3600
hw.sensors.cpu0.frequency0=36.00 Hz



Other thing that is interesting, is AES-NI on AMD cpus fast as on Intel
cpus?

Re: When IPSec destination 0.0.0.0/0, I cannot ping directly connected Interfaces

[Hrvoje Popovski]

On 12.3.2024. 17:11, Samuel Jayden wrote:
> Dear Misc,
> 
> I have an OpenBSD device with two interfaces: vport10 with an IP address of
> 192.168.83.1/24 and vport20 with an IP address of 192.168.85.1/24. I have
> configured IPSec to route all traffic from these two vport interfaces to
> another point through an IPSec tunnel using the destination network
> 0.0.0.0/0.
> 
> Due to IPSec operating before kernel routing, I cannot even ping the
> directly connected interfaces' IP addresses.
> 
> I've attempted to implement route-based PF rules to solve the issue, but
> unfortunately, the problem persists.
> I'm looking for a solution that allows for the local traffic between these
> two interfaces to bypass the IPSec tunnel, ensuring they can communicate
> with each other while keeping the IPSec destination network as 0.0.0.0/0.
> 
> Here's my IPSec configuration:
> 
> ike active esp tunnel from { 192.168.83.0/24 192.168.85.0/24 } to {
> 0.0.0.0/0 } \
> peer A.B.C.D \
> main auth hmac-md5 enc 3des group modp1024 lifetime 86400 \
> quick auth hmac-md5 enc 3des group none lifetime 43200 \
> psk "verysecret"
> 
> Thanks in advance.
> 

Hi,

put in ipsec.conf

flow from 192.168.83.0/24 to 192.168.83.0/24 type bypass
flow from 192.168.83.0/24 to 192.168.85.0/24 type bypass
flow from 192.168.85.0/24 to 192.168.85.0/24 type bypass
flow from 192.168.85.0/24 to 192.168.83.0/24 type bypass

and if you have carp than put this also

flow from 192.168.83.0/24 to 224.0.0.18/32 type bypass
flow from 192.168.85.0/24 to 224.0.0.18/32 type bypass

or something like that . .

Dell BOSS-S1 adapter or HBA330 non-raid

[Hrvoje Popovski]

Hi all,

did anyone installed and boot successfully OpenBSD on Dell BOSS-S1
adapter or HBA330 non-raid controller ?
I've got Dell R740xd in lab and of course for storage controllers there
are BOSS-S1 and HBA330. :)
OpenBSD can be installed on these controllers but unfortunately it panic
at boot.

I will send proper bug report to bugs@, but I would like to know if
someone have some experience with this controllers and OpenBSD.

Thank you...

Re: cvs revert specific commit

[Hrvoje Popovski]

On 19.1.2024. 0:14, Christian Weisgerber wrote:
> Hrvoje Popovski:
> 
>> I would like to revert only if_em.c rev. 1.369, but would like to leave
>> TSO stuff if_em.c rev. 1.370 and if_em.h rev 1.81.
>>
>> is this somehow possible?
> 
> $ cd /sys/dev/pci
> $ cvs diff -kk -r1.369 -r1.368 if_em.c | patch -p0
> 

Thank you.

cvs revert specific commit

[Hrvoje Popovski]

Hi all,

I sorry for beginners questions regarding cvs revert stuff.

https://cvsweb.openbsd.org/src/sys/dev/pci/if_em.c
https://cvsweb.openbsd.org/src/sys/dev/pci/if_em.h

I would like to revert only if_em.c rev. 1.369, but would like to leave
TSO stuff if_em.c rev. 1.370 and if_em.h rev 1.81.

is this somehow possible?

Re: upgrade to latest snapshot failing

[Hrvoje Popovski]

On 17.11.2023. 20:05, Stuart Henderson wrote:
> On 2023-11-17, Sonic  wrote:
>> Following -current:
>> OpenBSD 7.4-current (GENERIC.MP) #1447: Wed Nov 15 09:56:54 MST 2023
>> Upgrade via "sysupgrade -s" now failing with:
>> init: single user shell terminated, restarting
>> init: single user shell terminated, restarting
>> init: single user shell terminated, restarting
>> 
> 
> Bit old by now, boot bsd.rd and try a newer one?
> 
> If that fails too: which arch?
> 

Hi,

I've tried sysupgrade just now and I have same problem as Chris but it's
RAMDISK

boot>
booting hd0a:/bsd.upgrade: 3973830+1655808+3882648+0+704512
[109+445968+298155]=0xa75eb0
entry point at 0x81001000
 Copyright (c) 1982, 1986, 1989, 1991, 1993
The Regents of the University of California.  All rights reserved.
Copyright (c) 1995-2023 OpenBSD. All rights reserved.
https://www.OpenBSD.org

OpenBSD 7.4-current (RAMDISK_CD) #1374: Fri Nov 17 10:10:26 MST 2023
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/RAMDISK_CD
real mem = 17115840512 (16322MB)
avail mem = 16593100800 (15824MB)
random: good seed from bootblocks
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xcf42c000 (99 entries)
bios0: vendor Dell Inc. version "2.9.0" date 12/06/2019
bios0: Dell Inc. PowerEdge R620
acpi0 at bios0: ACPI 3.0
acpi0: tables DSDT FACP APIC SPCR HPET DMAR MCFG WDAT SLIC ERST HEST
BERT EINJ TCPA PC__ SRAT SSDT
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 4 (boot processor)
cpu0: Intel(R) Xeon(R) CPU E5-2643 v2 @ 3.50GHz, 3600.49 MHz, 06-3e-04,
patch 042e
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSEN
cpu0: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 256KB
64b/line 8-way L2 cache, 25MB 64b/le
cpu0: apic clock running at 100MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.1, IBE
cpu at mainbus0: not configured
cpu at mainbus0: not configured
cpu at mainbus0: not configured
cpu at mainbus0: not configured
cpu at mainbus0: not configured
ioapic0 at mainbus0: apid 0 pa 0xfec0, version 20, 24 pins
ioapic1 at mainbus0: apid 1 pa 0xfec3f000, version 20, 24 pins, remapped
acpihpet0 at acpi0: 14318179 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 1 (PEX1)
acpiprt2 at acpi0: bus -1 (PE1C)
acpiprt3 at acpi0: bus 3 (PEX2)
acpiprt4 at acpi0: bus 2 (PEX3)
acpiprt5 at acpi0: bus 4 (PEX4)
acpiprt6 at acpi0: bus -1 (PEX5)
acpiprt7 at acpi0: bus 8 (PEX6)
acpiprt8 at acpi0: bus 7 (PEX7)
acpipci0 at acpi0 PCI0: 0x 0x0011 0x0001
acpicmos0 at acpi0
com1 at acpi0 COMA addr 0x2f8/0x8 irq 3: ns16550a, 16 byte fifo
com1: console
com0 at acpi0 COMB addr 0x3f8/0x8 irq 4: ns16550a, 16 byte fifo
acpipci1 at acpi0 P0B1: 0x 0x0011 0x0001
acpipci2 at acpi0 P1B1: 0x 0x0011 0x0001
"PNP0C14" at acpi0 not configured
acpicpu at acpi0 not configured
cpu0: using VERW MDS workaround
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel E5 v2 Host" rev 0x04
ppb0 at pci0 dev 1 function 0 "Intel E5 v2 PCIE" rev 0x04
pci1 at ppb0 bus 1
1:0:1: rom address conflict 0xd800/0x8
ix0 at pci1 dev 0 function 0 "Intel 82599" rev 0x01, msix, 1 queue,
address ec:f4:bb:da:f7:f8
ix1 at pci1 dev 0 function 1 "Intel 82599" rev 0x01, msix, 1 queue,
address ec:f4:bb:da:f7:fa
ppb1 at pci0 dev 2 function 0 "Intel E5 v2 PCIE" rev 0x04: msi
pci2 at ppb1 bus 3
ppb2 at pci0 dev 2 function 2 "Intel E5 v2 PCIE" rev 0x04
pci3 at ppb2 bus 2
mfi0 at pci3 dev 0 function 0 "Symbios Logic MegaRAID SAS2008" rev 0x03:
apic 1 int 10
mfi0: "PERC H310 Mini", firmware 20.13.3-0001
scsibus0 at mfi0: 16 targets
scsibus1 at mfi0: 256 targets
sd0 at scsibus1 targ 0 lun 0: 
naa.55cd2e4150610e56
sd0: 228936MB, 512 bytes/sector, 468862128 sectors
ppb3 at pci0 dev 3 function 0 "Intel E5 v2 PCIE" rev 0x04: msi
pci4 at ppb3 bus 4
4:0:1: rom address conflict 0xda00/0x8
ix2 at pci4 dev 0 function 0 "Intel X540T" rev 0x01, msix, 1 queue,
address a0:36:9f:29:f2:0c
ix3 at pci4 dev 0 function 1 "Intel X540T" rev 0x01, msix, 1 queue,
address a0:36:9f:29:f2:0e
"Intel E5 v2 I/OAT" rev 0x04 at pci0 dev 4 function 0 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci0 dev 4 function 1 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci0 dev 4 function 2 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci0 dev 4 function 3 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci0 dev 4 function 4 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci0 dev 4 function 5 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci0 dev 4 function 6 not configured
"Intel E5 v2 I/OAT" rev 0x04 at pci0 dev 4 function 7 not configured
"Intel E5 v2 Address Map" rev 0x04 at pci0 dev 5 function 0 not configured
"Intel E5 v2 IIO RAS" rev 0x04 at pci0 dev 5 function 2 not configured
ppb4 at pci0 dev 17 function 0 "Intel C600 Virtual PCIE" rev 0x05
pci5 at ppb4 bus 5
"Intel C600 MEI" rev 0x05 at pci0 dev 22 function 0 not configured
"Int

Re: pf logging in ascii and send to remote syslog

[Hrvoje Popovski]

On 11.11.2023. 12:13, Stuart Henderson wrote:
> On 2023-11-11, Peter N. M. Hansteen  wrote:
>> On Fri, Nov 10, 2023 at 08:23:54PM +0100, Hrvoje Popovski wrote:
>>> what would be best way to log pf logs in ascii and sent it to remote
>>> syslog ? I'm aware of pflow but I need ascii pf logs on remote syslog
>>> server.
>>
>> something like the good old 
>> https://home.nuug.no/~peter/pf/newest/log2syslog.html
>> should still work, I think.
> 
> Or 
> https://cvsweb.openbsd.org/cgi-bin/cvsweb/~checkout~/www/faq/pf/logging.html?rev=1.68#syslog
> 
> If you don't need _all_ pf logs converting to syslog, you can create a
> separate interface "echo up | doas tee /etc/hostname.pflog1" and use
> "log to pflog1" on selected rules.
> 


Thank you Peter and Stuart that's exactly what I need ...

pf logging in ascii and send to remote syslog

[Hrvoje Popovski]

Hi all,

what would be best way to log pf logs in ascii and sent it to remote
syslog ? I'm aware of pflow but I need ascii pf logs on remote syslog
server.
I remember that it was on https://www.openbsd.org/faq/pf/logging.html
and that that section was removed.

Old version is on https://www.dragonflybsd.org/~aggelos/pf/logging.html

Is there maybe a better way how to do ascii logging besides what is in
the old version of logging.html?

Thank you.

Re: Greedy match of traffic in iked between site and hub

[Hrvoje Popovski]

On 15.10.2023. 18:56, Stuart Henderson wrote:
> On 2023-10-15, rea...@catastrophe.net  wrote:
>> What is a better way to configure iked on site-obsd so that it does not
>> encapsulate local traffic on the 10.89.2.0/24 network? Obviously my
>> understanding is incorrect, so any help is appreciated.
> 
> You should be able to add a bypass flow in ipsec.conf, and set ipsec=YES
> but *not* isakmpd_flags in rc.conf.local.
> 
> To load manually without rebooting, ipsecctl -f /etc/ipsec.conf
> 

Hi,

just to confirm what Stuart said...

I'm running firewall that routes 10.9/16 and that network needs to go
out through ipsec tunnel.

ike esp from 10.9.0.0/16 to any

beside 10.9/16 it routes other networks and because i have "10.9/16 to
any" i need to exclude traffic that originate from 10.9/16 to other
directly connected networks on that firewall ...


ipsec.conf

ike esp from 10.9.0.0/16 to any \
local X peer Y \

flow from 10.9.0.0/16 to 224.0.0.18/32 type bypass - this one is carp
flow from 10.9.0.0/16 to 10.9.0.0/16 type bypass - don't remember, but
it must be something

other directly connected network
flow from 10.9.0.0/16 to 10.8.0.0/16 type bypass
flow from 10.9.0.0/16 to 10.7.0.0/16 type bypass


ipsecctl -sa
flow esp in from 10.8.0.0/16 to 10.9.0.0/16 type bypass
flow esp in from 10.9.0.0/16 to 10.9.0.0/16 type bypass
flow esp in from 10.7.0.0/16 to 10.9.0.0/16 type bypass

flow esp out from 10.9.0.0/16 to 10.8.0.0/16 type bypass
flow esp out from 10.9.0.0/16 to 10.9.0.0/16 type bypass
flow esp out from 10.9.0.0/16 to 10.7.0.0/16 type bypass

Re: 7.4 and hostname.pfsync7

[Hrvoje Popovski]

On 15.10.2023. 6:51, Harald Dunkel wrote:
> Hi folks,
> 
> I learned that pfsync has been rewritten for 7.4 and that
> 
> up
> syncdev em7
> 
> doesn't work anymore. What about
> 
> up syncdev em7
> 
> (one line), as suggested in the current pfsync(4)?
> 
> 
> Regards
> Harri
> 

could you try

syncdev em7
up

Re: OpenBSD 7.2 fw stack trace on Dell R740

[Hrvoje Popovski]

On 26.9.2023. 9:24, Joerg Streckfuss wrote:
> 
> Hi Stuart,
> 
> Am 25.09.23 um 19:08 schrieb Stuart Henderson:
>> That might possibly be the one fixed by 7.2 errata 008, so if you don't
>> already have that you at least want to syspatch.
> 
> That was my guess as well. However, the systems were patched up to 7.2
> errata-016. I applied the remaining patches. So far the systems are
> running stable. Are there any changes between the 7.2 and 7.3 releases
> that could indicate a bug?
> 
> Many regards,
> 
> Joerg

Hi,

If you can go to snapshot or wait for 7.4. Thanks to dlg@ my pfsync
firewalls are rock stable after this commit

https://marc.info/?l=openbsd-cvs&m=168861927203498&w=2

Re: Stacked MTUs

[Hrvoje Popovski]

On 7.9.2023. 18:45, Lyndon Nerenberg (VE7TFX/VE6BBM) wrote:
> I'm setting up jumbograms on a couple of vlans stacked
> on an aggr and I need a sanithy check that I'm doing
> this right.
> 
> The switches use a hardware MTU of 9192.  We want an IP
> MTU of 9000 for the vlans.  I'm assuming this will work?
> 
>   ifconfig em1 mtu 9192
>   ifconfig em5 mtu 9192
>   ifconfig aggr0 9192  # em1+em5 lacp
>   ifconfig vlanX mtu 9000 # stacked on aggr0
>   ifconfig vlanY mtu 1500 # ditto
> 
> --lyndon
> 

can you send

dmesg | grep em

ifconfig em1 hwfeatures
ifconfig em5 hwfeatures

Re: Route based IPsec

[Hrvoje Popovski]

On 27.5.2023. 9:24, Valdrin MUJA wrote:
> Hello,
> 
> I need Route based IPsec solution to set up between a firewall device and 
> my OpenBSD firewall.
> However, I am a little confused about this:
> I created more than one enc device, I did policy based routing with PF but no 
> results. I guess this is not the intended use of interfaces like enc[0,1]. 
> But since I am not sure, I would to ask:
> Does OpenBSD have routed based IPsec support? Thanks in advance.
> 

little off topic ...if other side is aws ipsec gateway or vmware nsx,
then policy based ipsec is working quite nice, but yeah, route based
ipsec would be awesome

Re: supermicro 5019D-FTN4 server with AMD EPYC 3251 SoC Processor

[Hrvoje Popovski]

On 30.6.2021. 15:34, Denis Fondras wrote:
> Le Tue, Jun 29, 2021 at 07:46:55PM +0200, EdaSky a écrit :
>> Good day everyone
>>
>> Does anyone use supermicro 5019D-FTN4 server with AMD EPYC 3251 SoC
>> Processor?
>>
>> https://www.supermicro.com/Aplus/system/Embedded/AS-5019D-FTN4.cfm
>>
>> Experience and dmesg would be perfect.
>>
> 
> Experience is perfect so far. I am really happy with it as BGP edge.
> 
> 


Hi Denis,

are you still happy with this box?

Is amd64 OpenBSD stable on it? Are you happy with pf or forwarding
performance ?

As in your dmesg, I thought of putting 4 port ixl(4) card in it, 1G sfp
uplink, 10G for internal vlans and em(4) for pfsync...






> OpenBSD 6.9-current (GENERIC.MP) #20: Sun May 16 00:32:45 MDT 2021
> dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
> real mem = 34228760576 (32643MB)
> avail mem = 33175949312 (31639MB)
> random: good seed from bootblocks
> mpath0 at root
> scsibus0 at mpath0: 256 targets
> mainbus0 at root
> bios0 at mainbus0: SMBIOS rev. 2.8 @ 0xdab19000 (51 entries)
> bios0: vendor American Megatrends Inc. version "1.0c" date 06/30/2020
> bios0: Supermicro AS -5019D-FTN4
> acpi0 at bios0: ACPI 6.1
> acpi0: sleep states S0 S5
> acpi0: tables DSDT FACP APIC FPDT FIDT SSDT SPMI SSDT MCFG SSDT CRAT CDIT 
> BERT EINJ HEST HPET SSDT UEFI SSDT WSMT
> acpi0: wakeup devices S0D0(S3) S0D1(S3) S0D2(S3) S0D3(S3) S1D0(S3) S1D1(S3) 
> S1D2(S3) S1D3(S3)
> acpitimer0 at acpi0: 3579545 Hz, 32 bits
> acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
> cpu0 at mainbus0: apid 0 (boot processor)
> cpu0: AMD EPYC 3251 8-Core Processor, 2500.55 MHz, 17-01-02
> cpu0: 
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA,IBPB,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
> cpu0: 64KB 64b/line 4-way I-cache, 32KB 64b/line 8-way D-cache, 512KB 
> 64b/line 8-way L2 cache
> cpu0: ITLB 64 4KB entries fully associative, 64 4MB entries fully associative
> cpu0: DTLB 64 4KB entries fully associative, 64 4MB entries fully associative
> cpu0: smt 0, core 0, package 0
> mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
> cpu0: apic clock running at 99MHz
> cpu0: mwait min=64, max=64, C-substates=1.1, IBE
> cpu1 at mainbus0: apid 2 (application processor)
> cpu1: AMD EPYC 3251 8-Core Processor, 2500.01 MHz, 17-01-02
> cpu1: 
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA,IBPB,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
> cpu1: 64KB 64b/line 4-way I-cache, 32KB 64b/line 8-way D-cache, 512KB 
> 64b/line 8-way L2 cache
> cpu1: ITLB 64 4KB entries fully associative, 64 4MB entries fully associative
> cpu1: DTLB 64 4KB entries fully associative, 64 4MB entries fully associative
> cpu1: smt 0, core 1, package 0
> cpu2 at mainbus0: apid 4 (application processor)
> cpu2: AMD EPYC 3251 8-Core Processor, 2500.01 MHz, 17-01-02
> cpu2: 
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA,IBPB,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
> cpu2: 64KB 64b/line 4-way I-cache, 32KB 64b/line 8-way D-cache, 512KB 
> 64b/line 8-way L2 cache
> cpu2: ITLB 64 4KB entries fully associative, 64 4MB entries fully associative
> cpu2: DTLB 64 4KB entries fully associative, 64 4MB entries fully associative
> cpu2: smt 0, core 2, package 0
> cpu3 at mainbus0: apid 6 (application processor)
> cpu3: AMD EPYC 3251 8-Core Processor, 2500.01 MHz, 17-01-02
> cpu3: 
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA,IBPB,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
> cpu3: 64KB 64b/line 4-way I-cache, 32KB 64b/line 8-way D-cache, 512KB 
> 64b/line 8-way L2 cache
> cpu3: ITLB 64 4KB entries fully associative, 64 4MB entries fully associative
> cpu3: DTLB 64 4KB entries fully associative, 64 4MB entries fully a

Re: ixl not seeing SFP+ modules ?

[Hrvoje Popovski]

On 14.4.2023. 19:36, Laura Smith wrote:
> I have an ixl card (ixl0 at pci1 dev 0 function 0 "Intel X710 SFP+" rev 0x02: 
> port 3, FW 6.0.48442 API 1.7, msix, 4 queues) on OpenBSD that doesn't seem to 
> be seeing any of my SFP+ modules.
> 
> 
> The modules are all MSA coded and from different manufacturers.
> 
> 
> ifconfig ixl shows "status: no carrier" (but light is confirmed as existing 
> both ways and all patching has been triple checked).
> 
> Additionally "ifconfig ixl transciever" reports "ifconfig: transciever: bad 
> value" whilst I believe this should be showing transceiver stats ?
> 
> 

Hi,

try ifconfig ixl0 sff

if you have some other 10G card ix(4) x520 intel or bnxt, try put that
sfp+ in them, maybe there will work, or just buy intel compatible sfp+
from fs.com or similar stores ...

Re: Hardware RAID on Poweredge Servers

[Hrvoje Popovski]

On 30.3.2023. 18:33, Kihaguru Gathura wrote:
> Hello,
> 
> Is hardware RAID on Poweredge servers (T340, PERC H330 in particular)
> generally stable enough for production or is it safer to stick with OpenBSD
> softraid?
> 

Hi,

not sure if there is big differences between H330 and H330 mini but H330
mini is quite stable ..



bios0: Dell Inc. PowerEdge R630

mfii0 at pci1 dev 0 function 0 "Symbios Logic MegaRAID SAS3008" rev
0x02: msi
mfii0: "PERC H330 Mini", firmware 25.5.5.0005


uptime
 6:39PM  up 1205 days,  2:14, 1 user, load averages: 0.38, 0.40, 0.40


bioctl sd0
Volume  Status   Size Device
mfii0 0 Online   299439751168 sd0 RAID1 WT
  0 Online   3000 1:0.0   noencl 
  1 Online   3000 1:1.0   noencl 

Re: Using veb instead of bridge at vpls section

[Hrvoje Popovski]

On 20.3.2023. 20:05, Valdrin MUJA wrote:
> Hello folks,
> 
> I have successfully configured the VPLS by following the instruction on 
> https://pawa.lt/posts/2018/01/vpls-with-openbsd/.
> Everything worked like a charm.
> 
> But when I tried to use veb(4)  instead of bridge(4) , I got 'Device Busy' 
> error.
> I'm guessing ldpd(8) doesn't support the veb interface. Is it true?
> I'm just trying to be sure. If this is the case, I hope one day ldpd(8) will 
> get veb(4) support. Thanks for these great efforts.
> 

Hi,

maybe try to use tpmr(4) ?

>From man tpmr

An equivalent setup using MPLS pseudowires instead of IP as the
transport can be built using mpw(4) interfaces.

With tpmr pf is enabled by default you can disable it with -link1

Re: Selecting a 10G NIC

[Hrvoje Popovski]

On 17.2.2023. 18:29, Nicolas Goy wrote:
> I know this question has been answered multiple times, but I wonder if
> things changed with 7.2.
> 
> Which NIC would provide the best performance with 10G physical layer
> with open bsd?
> 
> I have choice between intel e810, x710, x550, x520, broadcom
> BCM957414A4142CC or maybe even something else.

go with x520 or x710. e810 is not supported and broadcom in my
experience is not that stable.

x520 can have up to 16 queues
x710 can have up to 8 queues but with power of 2

with or without pf and with standard imix traffic you could saturate 10G
if you have fast cores ... and by fast i mean amd fast, not intel fast :)

if you have pfsync forwarding will be slower
if you have ipsec tunnels forwarding will be much slower


long time ago i've stopped worrying about performance and start learning
about features that pf and openbsd gave me

Re: Performance optimizing OpenBSD 7.2

[Hrvoje Popovski]

On 15.2.2023. 10:28, Gábor LENCSE wrote:

> In OpenBSD, the packet forwarding happens single threaded, so the
> performance of your system does not benefit much from the 4 cores.

Hi,

actually if forwarding is single threaded of not, depends of what nic do
you have in box. ix,mcx,bnxt,igc,vmx and maybe others have multiqueue
support and with them you will have better forwarding performance. But
only forwarding performance, not better pf performance or ipsec or
something else...

If you have em(4) you can test em(4) multiqueue diff on tech@
https://marc.info/?l=openbsd-tech&m=165642186010149&w=2
but for now em doesn't have multiqueue which means that your forwarding
is single threaded.

For vmware if you have vmx(4) you will have multiqueue support and you
can see that in dmesg and with vmstat

obsd1# dmesg | grep vmx
vmx0 at pci11 dev 0 function 0 "VMware VMXNET3" rev 0x01: msix, 4
queues, address 00:0c:29:b6:ec:81
vmx1 at pci19 dev 0 function 0 "VMware VMXNET3" rev 0x01: msix, 4
queues, address 00:0c:29:b6:ec:8b

obsd1# vmstat -iz | grep vmx
irq114/vmx0 00
irq115/vmx0:0   327060
irq116/vmx0:1   126290
irq117/vmx0:2   123970
irq118/vmx0:3  220
irq123/vmx1 00
irq124/vmx1:0   60
irq125/vmx1:1   20
irq126/vmx1:2   00
irq127/vmx1:3   00


But for now there is problem on machines with lots of cpus and
multiqueue nics and that is number of interrupts.

https://marc.info/?l=openbsd-misc&m=167472080905265&w=2

Re: Intel nic on Dell R710: failed to allocate interrupt slot for PIC msix

[Hrvoje Popovski]

On 25.1.2023. 12:24, Joerg Streckfuss wrote:
> 
> Dear List,
> 
> we have problems with Intel nics of type Intel X710 (10 GbE) on a Dell
> R740. In total we have three nics with four ports each. With the uprade
> to OpenBSD 6.8 we lost two ports (ixl11 and ixl12). Now we upraded
> iteratively to OpenBSD 7.1 an we lost another port (ixl10). The update
> to OpenBSD 7.2 is pending, but I don't want to risk losing another port.
> 
> Cause seems to be a problem with the interrupt assignment.
> 
> The relevant dmesg part is as fallows:
> 
> 
> ixl11 at pci12 dev 0 function 1 "Intel X710 SFP+" rev 0x02: port 2, FW
> 7.83.59945 API 1.9, ms1
> failed to allocate interrupt slot for PIC msix pin -2135686911
> ixl11: unable to establish interrupt 1
> ixl12 at pci12 dev 0 function 2 "Intel X710 SFP+" rev 0x02: port 0, FW
> 7.83.59945 API 1.9, ms2
> failed to allocate interrupt slot for PIC msix pin -2135686655
> ixl12: unable to establish interrupt 1
> ixl13 at pci12 dev 0 function 3 "Intel X710 SFP+" rev 0x02: port 1, FW
> 7.83.59945 API 1.9, ms3
> failed to allocate interrupt slot for PIC msix pin -2135686399
> ixl13: unable to establish interrupt 1
> 
> 

Hi,

I have same problems on few firewalls.
One with 16 cores and 4 mcx,
second with 24 cores and bnxt,mcx,ixl and ix <- this on is in lab, so
not a big problem  :)
third on 12 core machine with 2 ix and 4 em but only when testing
jmatthew@ em multiqueue diff
https://marc.info/?l=openbsd-tech&m=165642186010149&w=2

Workaroud for this problem, at least for me, is to lower number of cpu
in BIOS or remove some nics. In my case, I lowered 16 cores to 12, then
mcx have 8 queues and that was fine for that firewall.
For you, maybe try to lower cpu's to 6, then ixl would have 4 queues,
and maybe, just maybe, everything will work for you :)
Just keep in mind that parallel forwarding is using 4 cpu, so at least
you need 4 cpu's or 4 queues on nic.

I think that this is nice insight and relevant answer from kettenis@
https://www.mail-archive.com/tech@openbsd.org/msg71790.html

I think that this problem isn't ignored by developers, but on other hand
more and more people is having this problem. When jmatthew@ em diff
would be commited, even more people would report this problem.



> full dmesg:
> 
> 
> Booting from Hard drive C:
> Using drive 0, partition 3.
> Loading..
> probing: pc0 com0 mem[624K 1266M 2M 398M 30720M a20=on]
> disk: hd0+
>>> OpenBSD/amd64 BOOT 3.53
> switching console to com0
>>> OpenBSD/amd64 BOOT 3.53
> booting hd0a:/bsd: 15639832+3699728+348192+0+1175552
> [1126995+128+1220904+924861]=0x17074c0
> entry point at 0x81001000
>  �[ using 3273920 bytes of bsd ELF
> symbol table ]
> Copyright (c) 1982, 1986, 1989, 1991, 1993
>     The Regents of the University of California.  All rights reserved.
> Copyright (c) 1995-2022 OpenBSD. All rights reserved. 
> https://www.OpenBSD.org
> 
> OpenBSD 7.1 (GENERIC.MP) #2: Fri Jan 20 13:16:22 MST 2023
>    
> t...@syspatch-71-amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
> real mem = 33941528576 (32369MB)
> avail mem = 32895590400 (31371MB)
> random: good seed from bootblocks
> mpath0 at root
> scsibus0 at mpath0: 256 targets
> mainbus0 at root
> bios0 at mainbus0: SMBIOS rev. 3.2 @ 0x68e36000 (75 entries)
> bios0: vendor Dell Inc. version "2.10.2" date 02/24/2021
> bios0: Dell Inc. PowerEdge R740
> acpi0 at bios0: ACPI 6.1
> acpi0: sleep states S0 S5
> acpi0: tables DSDT FACP SSDT TPM2 SSDT MCEJ WDAT SLIC HPET APIC MCFG
> MIGT MSCT PCAT PCCT RASFJ
> acpi0: wakeup devices XHCI(S4) RP17(S4) PXSX(S4) RP18(S4) PXSX(S4)
> RP19(S4) PXSX(S4) RP20(S4)]
> acpitimer0 at acpi0: 3579545 Hz, 24 bits
> acpihpet0 at acpi0: 2399 Hz
> acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
> cpu0 at mainbus0: apid 2 (boot processor)
> cpu0: Intel(R) Xeon(R) Gold 5122 CPU @ 3.60GHz, 3692.05 MHz, 06-55-04
> cpu0:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,N
> cpu0: 256KB 64b/line 8-way L2 cache
> cpu0: cannot disable silicon debug
> cpu0: smt 0, core 1, package 0
> mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
> cpu0: apic clock running at 24MHz
> cpu0: mwait min=64, max=64, C-substates=0.2.0.2, IBE
> cpu1 at mainbus0: apid 10 (application processor)
> cpu1: Intel(R) Xeon(R) Gold 5122 CPU @ 3.60GHz, 3691.34 MHz, 06-55-04
> cpu1:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,N
> cpu1: 256KB 64b/line 8-way L2 cache
> cpu1: cannot disable silicon debug
> cpu1: smt 0, core 5, package 0
> cpu2 at mainbus0: apid 4 (application processor)
> cpu2: Intel(R) Xeon(R) Gold 5122 CPU @ 3.60GHz, 3691.33 MHz, 06-55-04
> cpu2:
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,N
> cpu2: 256KB 64b/line 8-way L2 cache
> cpu2: cannot disable silicon debug
> cpu2: smt 0, core 2, package 0
> cpu3 at mainbus0: apid 22 (application processor)
> cpu3: Intel(R) Xeon

Re: do i need to move to veb?

[Hrvoje Popovski]

On 23.1.2023. 16:24, kasak wrote:
> 
> 22.01.2023 14:49, David Gwynne пишет:
>> On Sat, Jan 21, 2023 at 03:41:56PM +0300, kasak wrote:
>>> Hello misc!
>>>
>>> I'm using bridge for integrating remote clients to my network with this
>>> simple config:
>>>
>>> $ cat /etc/hostname.bridge0
>>> add vether0
>>> add em1
>>> add tap1
>>> up
>>>
>>> I see in this commit that veb is supposed to replace bridge
>>> https://marc.info/?l=openbsd-cvs&m=161405102019493&w=2
>>>
>>> Does it make sense to move to veb for me, or not?
>>> There is approximately 150 clients on the "em1" side and 10 on "tap1"
>> unless you're using pf to filter on em1 and tap1, then moving from
>> bridge and vether to veb and vport is simple. veb can be a lot faster
>> than bridge, so maybe that's a reason to try moving?
>>
>> dlg
>>
> I've followed your advice and failed :(
> 
> I moved hostname.bridge0 to hostname.veb0, moved hostname.vether0 to
> hostname.vport0
> 
> and edit hostname.veb0 replacing add vether0 to add vport0
> 
> after reboot i cannot reach veb0 network :( ping answer "the network is
> down"
> 
> 

Did you put "up" at the end of hostname.veb0 and maybe at the end of
hostname.vport0 ?

Re: veb(4) with multiple vlan(4)'s

[Hrvoje Popovski]

On 22.1.2023. 12:45, David Gwynne wrote:
>> hostname.veb1
> description "LAN"
> 
>> link1
> you don't want to enable link1 unless you want pf to filter traffic on
> the veb ports, and then you have to be careful to avoid having pf see
> the packet again on the vport1 interface.
> 

ah, yes, yes thank you ...
is because of that, that on tpmr(4) pf is enabled by default and on
veb(4) isn't?

Re: veb(4) with multiple vlan(4)'s

[Hrvoje Popovski]

On 22.1.2023. 3:27, Scott Colby wrote:
> Hello,
> 
> I am trying to set up a router with a fresh install of OpenBSD 7.2,
> and I'm having a hard time grokking how to use veb.
> 
> I have organized my network into 4 subnets:
> 
> - DHCP "WAN"
> - 192.168.0.0/24 "LAN"
> - 192.168.2.0/24 "IOT"
> - 192.168.3.0/24 "Guest"
> 
> My computer has 4 interfaces em{0..3} and my desired setup has the
> following qualities:
> - em0 is the WAN uplink with DHCP
> - em1 is the uplink to my WAP and carries all 3 internal networks,
>   with "LAN" untagged and "IOT" and "Guest" tagged as VLAN 1102
>   and 1103, respectively
> - em2 carries only "LAN", untagged
> - em3 carries only "IOT", untagged
> 
> I think I should have configuration files like:
> hostname.em0:
> inet autoconf
> 
> hostname.em{1..3}:
> up
> 
> hostname.veb0:
> add em1
> add em2
> add em3
> add vport0  # ??
> add vport1  # ??
> up
> 
> As for the vlan and vport interfaces, I have no idea.
> 
> After this, of course, I will want to do some filtering with pf
> (such as hosts on "IOT" and "Guest" not having access to hosts on
> "LAN.")
> 

Didn't test this but maybe something like this

hostname.em0
inet autoconf

hostname.em1
up

hostname.em2
up

hostname.em3
up

hostname.vport1
inet X.X.X.X/XX <- gateway for IOT

hostname.veb1
link1
add em1
add em2
add vport1
up

hostname.vlan1102
parent em1
vnetid 1102
up

hostname.vport2
address X.X.X.X/XX <- gateway for IOT

hostname.veb2
link1
add vlan1102
add em3
add vport2
up

hostname.vlan1103
parent em1
vnetid 1103
address X.X.X.X/XX <- gateway for Guest
up


if this is working than you can use pf to filter traffic between networks.

man veb
man ifconfig and search for VEB


> My questions are thus:
> 1) What is the proper network configuration to achieve the above
>goal?
> 2) What is the right way to filter packets transiting between subnets
>in this configuration? I see in the man page that the directionality
>of packets emerging from a veb to the network stack is not normal.
>I've seen things with adding groups to the interfaces, but not
>sure what that gets me that using interface names in pf.conf
>doesn't.
> 
> 
> Thanks in advance for any help that you can provide!
> 
> Scott
> 

Re: bridge(4) question new network setup

[Hrvoje Popovski]

On 20.1.2023. 20:09, patrick keshishian wrote:
> Hello,
> 
> I am trying get a new ISP setup working.  The Router is
> causing some pain.  There is a /28 public block assigned.
> The DSL router can't be configured in transparent bridge
> mode (they say).  It holds on to one of the /28 addresses.
> 
> The setup looks something like this:
> (and hopefully the ascii "art" remains intact from gmail)
> 
>( internet )
> |
> | [WAN IP]
>   +-o--+
>  / DSL ROUTER / <-- Transparent bridge mode NOT possible
> +-o--+
>   | [ one of /28 Public IPs = $dslgw_ip ]
>   |
>   |
>   | $ext
> +-o--+
> ||
> | OpenBSD/pf o--- ( rest of /28 Public IP network )
> || $dmz  (DMZ: httpd, smtpd, ...)
> +-o--+
>  $lan | [10.x.x.1]
>   |
> ( 10.x.x.x network )
> 
> 
> As far as networking goes, I need to be spoken to as if I'm
> a fledgling.
> 
> I want to do the obvious: use OpenBSD/pf(4) to:
>  - Filter traffic from $ext to $dmz
>  - Filter traffic from $dmz outbound
>  - Filter traffic from $lan (10.x.x.x) to $dmz
>  - NAT traffic from $lan (10.x.x.x) outbound to internet
> 
> 
> I'm bridge(4)-ing $ext and $dmz.  Which means I must give
> one of the /28 public IP addresses to either $ext or $dmz
> to be able to do:
> 
> # route add default $dslgw_ip
> 
> (!?)
> 
> Am I missing something?
> Is there a better way to configure things?
> 
> Thanks,
> --patrick
> 

Hi,

If your ext interface is in same subnet as that /28 from your ISP then
you could:

- use veb(4) to bridge ext, dmz and vport(4) interface and add default
route to dslgw_ip. vport is ip interface for veb

- or on ext interface put ip alias with ip addresses from /28 public
range and than do binat-to or nat-to in pf to hosts in dmz

or maybe i totally misunderstood you  :)

Re: BiDi sfp in ix

[Hrvoje Popovski]

On 9.1.2023. 15:21, Hrvoje Popovski wrote:
> On 5.1.2023. 18:43, Hrvoje Popovski wrote:
>> On 4.1.2023. 14:20, Ivo Chutkin wrote:
>>> On 2.1.2023 г. 16:58 ч., Hrvoje Popovski wrote:
>>>> On 28.12.2022. 20:21, Stuart Henderson wrote:
>>>>> On 2022-12-28, Hrvoje Popovski  wrote:
>>>>>> Hi all,
>>>>>>
>>>>>> I don't have much experience with BiDi sfp, so I'm asking you guys,
>>>>>> should openbsd ix work with 1G BiDi sfp.
>>>>> should do, yes.
>>>>>
>>>>> in case you're not aware, bidi transceivers come in different types,
>>>>> e.g.
>>>>> your MaxLink ML-S5531-20 transmits at 1550nm and receives at 1310nm, so
>>>>> must be paired with a transceiver that transmits at 1310nm and receives
>>>>> at 1550nm (e.g. the MaxLink model is ML-S3155-20) - do you have that?
>>>>>
>>>>> also, they should normally be used with single-mode fibre (due to how
>>>>> the bidi optics are coupled into the fibre they *can* also work with
>>>>> multimode fibre, though if you do that, insertion loss is high so
>>>>> distance is much more limited, plus it's even more sensitive to bending
>>>>> than usual in that case).
>>>>>
>>>>>
>>>> Hi,
>>>>
>>>> everything is fine regarding transceiver and fiber. I've played with it
>>>> for few days with my ISP and that BiDI sfp works on mikrotik
>>>> RB5009UG+S+IN and cisco 2960 switch. On aruba 2540 (allow unsupported
>>>> transceiver), ibm switch and openbsd ix(4) it won't work.
>>>>
>>>> I've ordered few BiDi sfp from fs.com and maybe my ISP will lend me
>>>> MaxLink sfp so I could test them in lab.
>>>>
>>>> Thank you Stuart for information ...
>>>>
>>> Hi Hrvoje,
>>>
>>> Can you try setting NIC to use speed 1G since it is SFP, not 10G SFP+
>>> module.
>>> My experience is that "media: Ethernet autoselect" not always work.
>>>
>> Hi,
>>
>> yes that's one of the problems. I couldn't set media to 1000baseLX and
>> ifconfig ix0 media only showed me autoselect, even when BiDi was
>> inserted into nic.
>>
>> Maybe that's problem with x552 but i didn't have x520 near me at that time.
>>
>> Good thing is that BiDi sfp's arrived and I will play with them.
>>
> Hi all,
> 
> It seems that ix(4) is having problems with 1G BiDi sfp. I've tested
> FS.COM 1G BiDi and they are working on mikrotik and some switches but on
> OpenBSD I'm getting "status: no carrier" what ever I do.

Hi all,

claudio@ suggested to try same setup with FreeBSD. So I've tried FreeBSD
and Linux and results are same, 1G BiDi from FS.COM coded for Cisco
won't work with Intel 82599 10G card.
If I boot FreeBSD and Linux with that sfp in 82599 or X710 card that
interface will be disabled at boot and won't show up in OS.
On OpenBSD i could see it but mac address is 00:00:00:00:00:00

I've tried all combinations with unsupported_sfp statements for FreeBSD
and Linux but I couldn't see that interface in OS.
If I put that sfp in card after boot status is no carrier whatever I do,
at least on FreeBSD. I'm still playing with Linux..


it seems that I've missed out all football here :)

it seems that I've missed out all football here

Re: BiDi sfp in ix

[Hrvoje Popovski]

On 9.1.2023. 15:21, Hrvoje Popovski wrote:
> On 5.1.2023. 18:43, Hrvoje Popovski wrote:
>> On 4.1.2023. 14:20, Ivo Chutkin wrote:
>>> On 2.1.2023 г. 16:58 ч., Hrvoje Popovski wrote:
>>>> On 28.12.2022. 20:21, Stuart Henderson wrote:
>>>>> On 2022-12-28, Hrvoje Popovski  wrote:
>>>>>> Hi all,
>>>>>>
>>>>>> I don't have much experience with BiDi sfp, so I'm asking you guys,
>>>>>> should openbsd ix work with 1G BiDi sfp.
>>>>>
>>>>> should do, yes.
>>>>>
>>>>> in case you're not aware, bidi transceivers come in different types,
>>>>> e.g.
>>>>> your MaxLink ML-S5531-20 transmits at 1550nm and receives at 1310nm, so
>>>>> must be paired with a transceiver that transmits at 1310nm and receives
>>>>> at 1550nm (e.g. the MaxLink model is ML-S3155-20) - do you have that?
>>>>>
>>>>> also, they should normally be used with single-mode fibre (due to how
>>>>> the bidi optics are coupled into the fibre they *can* also work with
>>>>> multimode fibre, though if you do that, insertion loss is high so
>>>>> distance is much more limited, plus it's even more sensitive to bending
>>>>> than usual in that case).
>>>>>
>>>>>
>>>>
>>>> Hi,
>>>>
>>>> everything is fine regarding transceiver and fiber. I've played with it
>>>> for few days with my ISP and that BiDI sfp works on mikrotik
>>>> RB5009UG+S+IN and cisco 2960 switch. On aruba 2540 (allow unsupported
>>>> transceiver), ibm switch and openbsd ix(4) it won't work.
>>>>
>>>> I've ordered few BiDi sfp from fs.com and maybe my ISP will lend me
>>>> MaxLink sfp so I could test them in lab.
>>>>
>>>> Thank you Stuart for information ...
>>>>
>>> Hi Hrvoje,
>>>
>>> Can you try setting NIC to use speed 1G since it is SFP, not 10G SFP+
>>> module.
>>> My experience is that "media: Ethernet autoselect" not always work.
>>>
>>
>> Hi,
>>
>> yes that's one of the problems. I couldn't set media to 1000baseLX and
>> ifconfig ix0 media only showed me autoselect, even when BiDi was
>> inserted into nic.
>>
>> Maybe that's problem with x552 but i didn't have x520 near me at that time.
>>
>> Good thing is that BiDi sfp's arrived and I will play with them.
>>
> 
> Hi all,
> 
> It seems that ix(4) is having problems with 1G BiDi sfp. I've tested
> FS.COM 1G BiDi and they are working on mikrotik and some switches but on
> OpenBSD I'm getting "status: no carrier" what ever I do.
> I've tried:
> advertise 1G without auto-negotiation and speed 1Gbps full duplex
> advertise 1G with auto-negotiation
> With those sfp's in OpenBSD I just can't disable auto-neg or configure
> media to advertise 1G full-duplex.
> With normal 1G sfp everything is working as expected on OpenBSD ix(4).
> 
> For link to be UP between network equipment with 1G BiDi i needed to
> configure interfaces to advertise 1G without auto-negotiation and speed
> 1Gbps full duplex.
> 
> On other hand 10G BiDi is working as expected with auto negotiation,
> means that I didn't need to configure anything for link to be up.
> 
> 
> 
> 1G BiDi
> OpenBSD
> ix0: flags=8843 mtu 1500
> lladdr a0:36:9f:2e:96:a0
> index 1 priority 0 llprio 3
> media: Ethernet autoselect
> status: no carrier
> transceiver: SFP LC, 1490 nm, 10.0km SMF
> model: FS SFP-GE-BX rev A0
> serial: F2130238999, date: 2022-03-28
> voltage: 3.28 V, bias current: 12.90 mA
> temp: 41.18 C (low -10.00 C, high 80.00 C)
> tx: -6.42 dBm (low -10.00 dBm, high -2.00 dBm)
> rx: -7.24 dBm (low -23.98 dBm, high -2.00 dBm)
> inet 10.255.1.3 netmask 0xff00 broadcast 10.255.1.255
> ix0: flags=8843 mtu 1500
> lladdr a0:36:9f:2e:96:a0
> index 1 priority 0 llprio 3
> media: Ethernet autoselect
> status: no carrier
> supported media:
> media autoselect
> inet 10.255.1.3 netmask 0xff00 broadcast 10.255.1.255
> 
> 
> mikrotik
> name: sfp-sfpplus1
>   status: no-link
>   sfp-module-present: yes
>  sfp-rx-loss: no
> sfp-tx-fault: no
> sfp-type: SFP-or-SF

Re: BiDi sfp in ix

[Hrvoje Popovski]

On 9.1.2023. 16:39, Boyd Stephens wrote:
> Hrvoje,
> 
> I may be inquiring about an item that you have already provided but
> would it be possible for you to supply a copy of your hostname.ix0
> config file.  I have been unable to locate this bit of info while
> perusing this particular thread.
> 
> Thank you much.
> 
> ---
> Boyd
> 

Hi,

there's nothing special in hostname.ix0 file.

cat /etc/hostname.ix0
inet 10.255.1.3/24

interesting ifconfig commands for troubleshooting this kind of problems:
ifconfig ix0 transceiver or sff
ifconfig ix0 media

dmesg | grep ix
ix0 at pci3 dev 0 function 0 "Intel 82599" rev 0x01, msix, 12 queues
ix1 at pci3 dev 0 function 1 "Intel 82599" rev 0x01, msix, 12 queues

Re: BiDi sfp in ix

[Hrvoje Popovski]

On 5.1.2023. 18:43, Hrvoje Popovski wrote:
> On 4.1.2023. 14:20, Ivo Chutkin wrote:
>> On 2.1.2023 г. 16:58 ч., Hrvoje Popovski wrote:
>>> On 28.12.2022. 20:21, Stuart Henderson wrote:
>>>> On 2022-12-28, Hrvoje Popovski  wrote:
>>>>> Hi all,
>>>>>
>>>>> I don't have much experience with BiDi sfp, so I'm asking you guys,
>>>>> should openbsd ix work with 1G BiDi sfp.
>>>>
>>>> should do, yes.
>>>>
>>>> in case you're not aware, bidi transceivers come in different types,
>>>> e.g.
>>>> your MaxLink ML-S5531-20 transmits at 1550nm and receives at 1310nm, so
>>>> must be paired with a transceiver that transmits at 1310nm and receives
>>>> at 1550nm (e.g. the MaxLink model is ML-S3155-20) - do you have that?
>>>>
>>>> also, they should normally be used with single-mode fibre (due to how
>>>> the bidi optics are coupled into the fibre they *can* also work with
>>>> multimode fibre, though if you do that, insertion loss is high so
>>>> distance is much more limited, plus it's even more sensitive to bending
>>>> than usual in that case).
>>>>
>>>>
>>>
>>> Hi,
>>>
>>> everything is fine regarding transceiver and fiber. I've played with it
>>> for few days with my ISP and that BiDI sfp works on mikrotik
>>> RB5009UG+S+IN and cisco 2960 switch. On aruba 2540 (allow unsupported
>>> transceiver), ibm switch and openbsd ix(4) it won't work.
>>>
>>> I've ordered few BiDi sfp from fs.com and maybe my ISP will lend me
>>> MaxLink sfp so I could test them in lab.
>>>
>>> Thank you Stuart for information ...
>>>
>> Hi Hrvoje,
>>
>> Can you try setting NIC to use speed 1G since it is SFP, not 10G SFP+
>> module.
>> My experience is that "media: Ethernet autoselect" not always work.
>>
> 
> Hi,
> 
> yes that's one of the problems. I couldn't set media to 1000baseLX and
> ifconfig ix0 media only showed me autoselect, even when BiDi was
> inserted into nic.
> 
> Maybe that's problem with x552 but i didn't have x520 near me at that time.
> 
> Good thing is that BiDi sfp's arrived and I will play with them.
> 

Hi all,

It seems that ix(4) is having problems with 1G BiDi sfp. I've tested
FS.COM 1G BiDi and they are working on mikrotik and some switches but on
OpenBSD I'm getting "status: no carrier" what ever I do.
I've tried:
advertise 1G without auto-negotiation and speed 1Gbps full duplex
advertise 1G with auto-negotiation
With those sfp's in OpenBSD I just can't disable auto-neg or configure
media to advertise 1G full-duplex.
With normal 1G sfp everything is working as expected on OpenBSD ix(4).

For link to be UP between network equipment with 1G BiDi i needed to
configure interfaces to advertise 1G without auto-negotiation and speed
1Gbps full duplex.

On other hand 10G BiDi is working as expected with auto negotiation,
means that I didn't need to configure anything for link to be up.



1G BiDi
OpenBSD
ix0: flags=8843 mtu 1500
lladdr a0:36:9f:2e:96:a0
index 1 priority 0 llprio 3
media: Ethernet autoselect
status: no carrier
transceiver: SFP LC, 1490 nm, 10.0km SMF
model: FS SFP-GE-BX rev A0
serial: F2130238999, date: 2022-03-28
voltage: 3.28 V, bias current: 12.90 mA
temp: 41.18 C (low -10.00 C, high 80.00 C)
tx: -6.42 dBm (low -10.00 dBm, high -2.00 dBm)
rx: -7.24 dBm (low -23.98 dBm, high -2.00 dBm)
inet 10.255.1.3 netmask 0xff00 broadcast 10.255.1.255
ix0: flags=8843 mtu 1500
lladdr a0:36:9f:2e:96:a0
index 1 priority 0 llprio 3
media: Ethernet autoselect
status: no carrier
supported media:
media autoselect
inet 10.255.1.3 netmask 0xff00 broadcast 10.255.1.255


mikrotik
name: sfp-sfpplus1
  status: no-link
  sfp-module-present: yes
 sfp-rx-loss: no
sfp-tx-fault: no
sfp-type: SFP-or-SFP+
  sfp-connector-type: LC
  sfp-link-length-sm: 10km
 sfp-vendor-name: FS
  sfp-vendor-part-number: SFP-GE-BX
 sfp-vendor-revision: A0
   sfp-vendor-serial: F2040345575
  sfp-manufacturing-date: 22-04-06
  sfp-wavelength: 1310nm
 sfp-temperature: 34C
  sfp-supply-voltage: 3.265V
 sfp-tx-bias-current: 9mA
sfp-tx-power: -6.333dBm
sfp-rx-power: -6.203dBm
 eeprom-checksum: good




10G BiDi
OpenBSD
ix0: flags=8843 mtu 1500
lladdr a0:36:9f:2e:96

Re: BiDi sfp in ix

[Hrvoje Popovski]

On 4.1.2023. 14:20, Ivo Chutkin wrote:
> On 2.1.2023 г. 16:58 ч., Hrvoje Popovski wrote:
>> On 28.12.2022. 20:21, Stuart Henderson wrote:
>>> On 2022-12-28, Hrvoje Popovski  wrote:
>>>> Hi all,
>>>>
>>>> I don't have much experience with BiDi sfp, so I'm asking you guys,
>>>> should openbsd ix work with 1G BiDi sfp.
>>>
>>> should do, yes.
>>>
>>> in case you're not aware, bidi transceivers come in different types,
>>> e.g.
>>> your MaxLink ML-S5531-20 transmits at 1550nm and receives at 1310nm, so
>>> must be paired with a transceiver that transmits at 1310nm and receives
>>> at 1550nm (e.g. the MaxLink model is ML-S3155-20) - do you have that?
>>>
>>> also, they should normally be used with single-mode fibre (due to how
>>> the bidi optics are coupled into the fibre they *can* also work with
>>> multimode fibre, though if you do that, insertion loss is high so
>>> distance is much more limited, plus it's even more sensitive to bending
>>> than usual in that case).
>>>
>>>
>>
>> Hi,
>>
>> everything is fine regarding transceiver and fiber. I've played with it
>> for few days with my ISP and that BiDI sfp works on mikrotik
>> RB5009UG+S+IN and cisco 2960 switch. On aruba 2540 (allow unsupported
>> transceiver), ibm switch and openbsd ix(4) it won't work.
>>
>> I've ordered few BiDi sfp from fs.com and maybe my ISP will lend me
>> MaxLink sfp so I could test them in lab.
>>
>> Thank you Stuart for information ...
>>
> Hi Hrvoje,
> 
> Can you try setting NIC to use speed 1G since it is SFP, not 10G SFP+
> module.
> My experience is that "media: Ethernet autoselect" not always work.
> 

Hi,

yes that's one of the problems. I couldn't set media to 1000baseLX and
ifconfig ix0 media only showed me autoselect, even when BiDi was
inserted into nic.

Maybe that's problem with x552 but i didn't have x520 near me at that time.

Good thing is that BiDi sfp's arrived and I will play with them.

Re: BiDi sfp in ix

[Hrvoje Popovski]

On 28.12.2022. 20:21, Stuart Henderson wrote:
> On 2022-12-28, Hrvoje Popovski  wrote:
>> Hi all,
>>
>> I don't have much experience with BiDi sfp, so I'm asking you guys,
>> should openbsd ix work with 1G BiDi sfp.
> 
> should do, yes.
> 
> in case you're not aware, bidi transceivers come in different types, e.g.
> your MaxLink ML-S5531-20 transmits at 1550nm and receives at 1310nm, so
> must be paired with a transceiver that transmits at 1310nm and receives
> at 1550nm (e.g. the MaxLink model is ML-S3155-20) - do you have that?
> 
> also, they should normally be used with single-mode fibre (due to how
> the bidi optics are coupled into the fibre they *can* also work with
> multimode fibre, though if you do that, insertion loss is high so
> distance is much more limited, plus it's even more sensitive to bending
> than usual in that case).
> 
> 

Hi,

everything is fine regarding transceiver and fiber. I've played with it
for few days with my ISP and that BiDI sfp works on mikrotik
RB5009UG+S+IN and cisco 2960 switch. On aruba 2540 (allow unsupported
transceiver), ibm switch and openbsd ix(4) it won't work.

I've ordered few BiDi sfp from fs.com and maybe my ISP will lend me
MaxLink sfp so I could test them in lab.

Thank you Stuart for information ...

BiDi sfp in ix

[Hrvoje Popovski]

Hi all,

I don't have much experience with BiDi sfp, so I'm asking you guys,
should openbsd ix work with 1G BiDi sfp.


Thank you.



ix0 at pci5 dev 0 function 0 "Intel X552 SFP+" rev 0x00, msix, 4 queues,
ix1 at pci5 dev 0 function 1 "Intel X552 SFP+" rev 0x00, msix, 4 queues,

ifconfig ix0 media
ix0: flags=8843 rdomain 1 mtu 1500
lladdr ac:1f:6b:1c:db:9a
index 1 priority 0 llprio 3
media: Ethernet autoselect
status: no carrier
supported media:
media autoselect


ifconfig ix0 sff
ix0: flags=8843 rdomain 1 mtu 1500
lladdr ac:1f:6b:1c:db:9a
index 1 priority 0 llprio 3
media: Ethernet autoselect
status: no carrier
transceiver: SFP LC, 1550 nm, 20.0km SMF
model: MaxLink ML-S5531-20 rev A
serial: CIB2106070757, date: 2021-06-12
voltage: 3.28 V, bias current: 14.87 mA
temp: 35.72 C (low -45.00 C, high 90.00 C)
tx: -5.67 dBm (low -10.00 dBm, high -2.00 dBm)
rx: -5.67 dBm (low -23.98 dBm, high 0.00 dBm)



https://www.maxlink.eu/en/maxlink-125g-sfp-optical-module-wdmbidi-sm-tx-1550rx1310nm-20km-1x-lc-connector-ddm-65743/product

Re: poor routing/nat performance

[Hrvoje Popovski]

On 19.12.2022. 17:35, David Hajes wrote:
> hi guys,
> 
> I have simple PcEngines APU2 router running latest OpenBSD stable.
> 
> em0 is WAN (bridge to CaTV modem with 1Gbps/100Mbps connectivity with normal 
> ether connectivity with DHCP...no special stuff like PPPoE)
> 
> em1-3 is in vether/bridge mode with NAT routing to local network.
> 
> I have complained to ISP about speeds because it supposes to run almost 1Gbps.
> 
> results (speedtest.net used by ISP for some reason):
> 
> 800+/85 Mbps measured by ISP technician directly from CaTV modem.
> 440MBps/85Mbps simple NAT firewall pf.conf based on OpenBSD suggestions
> 380/80Mbps with my strict firewall rules
> 
> I have used following guide 
> http://dant.net.ru/calomel/network_performance.html No changes, same 
> performance.
> 
> Checking out router monitoring
> 
> 3k packets/s firewall throughput
> pf_states lookup max. 12k/s, ~2k/s
> CPU bored, max. load 25%
> RAM 2.6 GB from 4GB free, swap never used
> 
> I am guessing HW is not issue.
> 
> Is there any issues with bridging local interfaces, and routing/NAT 
> performance, please?
> 
> I tried to Google answers, and there is lots of whining but no real info. It 
> supposes to run double speed, at least 800Mbps as shown by ISP technicians.
> 
> Any suggestions for bottleneck, please?
> 

Could you try veb(4) instead bridge(4) ?
Bridge is quite slow

https://undeadly.org/cgi?action=article;sid=20220319123157

Re: Stretch/L2VPN between two datacenters

[Hrvoje Popovski]

On 16.12.2022. 11:33, Lars Bonnesen wrote:
> We are about to migrate VM's from one datacenter to another and the VMware
> L2VPN we are using for this is simply not stable for some reason that we
> cannot figure out why.
> 
> I have used GRE-tunneling before on a software router that I actually
> cannot remember the name of now, but if OpenBSD can do the same, I would
> rather deploy one OpenBSD on each site and have that task handled by
> OpenBSD.
> 
> Each site should be able to use the other site gateway over a
> L2-network.and VMs on each site should be able to see each other as they
> are on the same LAN
> 
> Where to start reading?


man tpmr

Solidrun - Bedrock

[Hrvoje Popovski]

Hi all,

I know that this box is new and can't be bought yet, only get for
evaluation but maybe someone have dmesg? :)
It looks very interesting to me.


https://www.solid-run.com/fanless-computers/industrial-embedded-computers/bedrock-v3000-basic/

https://www.servethehome.com/solidrun-bedrock-pc-with-the-amd-ryzen-v3000-series-coming/

Re: Does OpenBSD support Receive Side Scaling (also called: multi-queue receiving)

[Hrvoje Popovski]

On 15.10.2022. 9:39, Stuart Henderson wrote:
> On 2022-10-14, Gabor LENCSE  wrote:
>> Dear All,
>>
>> I am a researcher and I would like to benchmark the stateful NAT64 
>> performance of OpenBSD PF.
>>
>> I use a 32-core server as DUT (Device Under Test). When I use Linux for 
>> benchmarking other stateful NAT64 implementations, I use the "ethtool -N 
>> enp5s0f1 rx-flow-hash udp4 sdfn" command to include also the source and 
>> destination port numbers (not only the source and destination IP 
>> addresses) into the hash function to distribute the interrupts caused by 
>> packet arrivals evenly among all the CPU cores.
>>
>> I tried to find a similar solution under OpenBSD, but I could not. (I 
>> used search expressions like: OpenBSD RSS receive side scaling multi 
>> queue receiving) Perhaps it is called differently under OpenBSD, or 
>> maybe there is no such solution at all?
>>
>> Could you advise me please?
> 
> A few network drivers have support for multiple queues (if my grepping
> is correct: aq igc bnxt ix ixl mcx vmx) - typically you will see the
> nunber of queues reported in the dmesg attach line if supported - but
> there's no interface to adjust what's fed into the hash function.
> 
> 32 cores is quite a lot for OpenBSD, more than around 8 is likely to
> be a waste for current versions in many use cases.
> 


Hi,

does it make sense to mention RSS and other stuff like TSO, MSI-X,
Multiple queues in man ?

Something like
https://leaf.dragonflybsd.org/cgi/web-man?command=ix§ion=ANY

Re: AMD EPYC

[Hrvoje Popovski]

On 28.9.2022. 10:05, Kapetanakis Giannis wrote:
> Hi,
> 
> Looking for upgrading our firewall/router and thinking about switching from 
> Xeon to EPYC (73F3 - 16C @ 3.5 GHz).
> 
> Anyone running on EPYC? Any problems?
> 
> Alternative would be something like dual Intel Xeon Gold 5315Y - 8C @ 3.20
> 
> thanks,
> 
> Giannis
> 

Hi,

I'm running openbsd on

Supermicro AS-1114S-WTRT with
AMD EPYC 7413 24-Core Processor, 2650.00 MHz, 19-01-01


Dell PowerEdge R6515 with
AMD EPYC 7313P 16-Core Processor, 2994.38 MHz, 19-01-01
this one will be my new firewall


from time to time on:
Dell PowerEdge R7515 with
AMD EPYC 7702P 64-Core Processor, 1996.28 MHz, 17-31-00


I have Lenovo Thinkpad E14 gen2 with
AMD Ryzen 5 4500U with Radeon Graphics, 2370.55 MHz, 17-60-01


And all those boxes are working as you would expected.


Here's hw.sensors for 7313P when idle
alt-fw1# sysctl hw.sensors | grep freq
hw.sensors.cpu0.frequency0=18.00 Hz
hw.sensors.cpu1.frequency0=18.00 Hz
hw.sensors.cpu2.frequency0=185000.00 Hz
hw.sensors.cpu3.frequency0=18.00 Hz
hw.sensors.cpu4.frequency0=18.00 Hz
hw.sensors.cpu5.frequency0=18.00 Hz
hw.sensors.cpu6.frequency0=18.00 Hz
hw.sensors.cpu7.frequency0=18.00 Hz
hw.sensors.cpu8.frequency0=18.00 Hz
hw.sensors.cpu9.frequency0=175000.00 Hz
hw.sensors.cpu10.frequency0=18.00 Hz
hw.sensors.cpu11.frequency0=18.00 Hz
hw.sensors.cpu12.frequency0=18.00 Hz
hw.sensors.cpu13.frequency0=18.00 Hz
hw.sensors.cpu14.frequency0=18.00 Hz
hw.sensors.cpu15.frequency0=18.00 Hz


when doing stress -c 16
alt-fw1# sysctl hw.sensors | grep freq
hw.sensors.cpu0.frequency0=37.00 Hz
hw.sensors.cpu1.frequency0=37.00 Hz
hw.sensors.cpu2.frequency0=37.00 Hz
hw.sensors.cpu3.frequency0=37.00 Hz
hw.sensors.cpu4.frequency0=37.00 Hz
hw.sensors.cpu5.frequency0=37.00 Hz
hw.sensors.cpu6.frequency0=37.00 Hz
hw.sensors.cpu7.frequency0=37.00 Hz
hw.sensors.cpu8.frequency0=37.00 Hz
hw.sensors.cpu9.frequency0=37.00 Hz
hw.sensors.cpu10.frequency0=37.00 Hz
hw.sensors.cpu11.frequency0=37.00 Hz
hw.sensors.cpu12.frequency0=37.00 Hz
hw.sensors.cpu13.frequency0=37.00 Hz
hw.sensors.cpu14.frequency0=37.00 Hz
hw.sensors.cpu15.frequency0=37.00 Hz


Regarding networking, few days ago I've rediscover that if you have cpu
with 16 or more core's and 4 nic's that support 16 queues (mcx or ix) if
you enable all of them box freeze and you need to lower to 12 cores.
I'm playing with this problem right now and will send it to tech@ or bugs@

softnet em weirdness

[Hrvoje Popovski]

Hi all,

I'm testing forwarding over em with plain with snapshot

em0 at pci7 dev 0 function 0 "Intel 82576" rev 0x01: msi,
em1 at pci7 dev 0 function 1 "Intel 82576" rev 0x01: msi,
em2 at pci8 dev 0 function 0 "Intel I210" rev 0x03: msi,
em3 at pci9 dev 0 function 0 "Intel I210" rev 0x03: msi,
em4 at pci12 dev 0 function 0 "Intel I350" rev 0x01: msi,
em5 at pci12 dev 0 function 1 "Intel I350" rev 0x01: msi,
em6 at pci12 dev 0 function 2 "Intel I350" rev 0x01: msi,
em7 at pci12 dev 0 function 3 "Intel I350" rev 0x01: msi,
ix0 at pci5 dev 0 function 0 "Intel X552 SFP+" rev 0x00, msix, 4 queues,
ix1 at pci5 dev 0 function 1 "Intel X552 SFP+" rev 0x00, msix, 4 queues,

em0 to em5 is for testing, em6 is for ssh and other interfaces are in shut.
em0 and em1 interfaces are pci card and em2-em5 are onboard.

I'm sending traffic on all 6 em interfaces at the same time and when
doing that I'm seeing 4 active softnet task

sleep/2   bored 1:12 44.73% softnet
sleep/1   bored 2:55 35.99% softnet
sleep/1   bored 0:18 31.20% softnet
sleep/0   bored 0:05  5.86% softnet

is this ok ? I thought that parallel forwarding is active only over
multiqueue interfaces.


Other funny thing is that if I leave traffic for some time I can get
softnet tasks over 100%.

13281   469454  6400K  860K onproc/3  -   554:59 5071.39%
softnet
53429   575885  6400K  860K onproc/0  -88:27 4955.62%
softnet
57655   540258  1000K  860K sleep/2   bored   129:23 4683.54%
softnet
61778   462853  6400K  860K onproc/1  -   503:16 2583.94%
softnet
70216   420909 105   200K  860K sleep/2   pgzero6:48 489.60%
zerothread
22474   596339 -1800K  860K sleep/2   reaper6:47  6.15% reaper
56297   350215  2800K  860K run/0 - 0:12  1.76%
softclock
73318   605906  1000K  860K sleep/1   bored 0:02  1.46% systq


I am aware that this is extreme but I've never seen something like that,
so I'm posting it here if someone could give me rough explanation what
I'm seeing here ..

Thank you




OpenBSD 7.2-beta (GENERIC.MP) #692: Mon Aug 15 11:36:43 MDT 2022
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 17052663808 (16262MB)
avail mem = 16518471680 (15753MB)
random: good seed from bootblocks
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.8 @ 0xed9b0 (48 entries)
bios0: vendor American Megatrends Inc. version "2.3" date 05/07/2021
bios0: Supermicro Super Server
acpi0 at bios0: ACPI 5.0
acpi0: sleep states S0 S4 S5
acpi0: tables DSDT FACP APIC FPDT FIDT SPMI MCFG UEFI DBG2 HPET WDDT
SSDT SSDT SSDT PRAD DMAR HEST BERT ERST EINJ
acpi0: wakeup devices IP2P(S4) EHC1(S4) EHC2(S4) RP07(S4) RP08(S4)
BR1A(S4) BR1B(S4) BR2A(S4) BR2B(S4) BR2C(S4) BR2D(S4) BR3A(S4) BR3B(S4)
BR3C(S4) BR3D(S4) RP01(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz, 2200.31 MHz, 06-56-03
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu0: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 256KB
64b/line 8-way L2 cache, 6MB 64b/line 12-way L3 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.2, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz, 2200.01 MHz, 06-56-03
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,RDSEED,ADX,SMAP,PT,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu1: 32KB 64b/line 8-way D-cache, 32KB 64b/line 8-way I-cache, 256KB
64b/line 8-way L2 cache, 6MB 64b/line 12-way L3 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz, 2200.01 MHz, 06-56-03
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,

Re: Fanless amd64 sytem recommendations

[Hrvoje Popovski]

On 8.8.2022. 14:16, Rachel Roch wrote:
> My personal preference are Deciso boxes 
> (https://www.deciso.com/product-catalog/dec600/)
> 
> They come with OpenSense but  you can plug in a USB serial cable and install 
> OpenBSD with zero issues.
> 

Hi,

I would recommend to go with at least 4 cores and em(4) interfaces, i350
or i210. 4 cores because forwarding is parallel with 4 threads and maybe
in near future em(4) will be multiqueue ..

https://marc.info/?l=openbsd-tech&m=165642186010149&w=2

Re: Latest -current boots very slow in VM

[Hrvoje Popovski]

On 2.7.2022. 0:11, Mischa wrote:
> Hi All,
> 
> Just updated one of my -current test VMs to the snapshot of June 30.
> The boot process takes extremely long. As soon as it's booting:

Hi,

update to latest snaphost and console output will be fast again :)

OpenBSD 7.1-current (GENERIC.MP) #599: Fri Jul  1 22:10:16 MDT 2022
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP

Re: Cron running at 99% CPU for seemingly no reason

[Hrvoje Popovski]

On 15.5.2022. 16:56, Todd C. Miller wrote:
> On Sun, 15 May 2022 16:02:03 +0200, Hrvoje Popovski wrote:
> 
>> I know how to rebuild cron
>>
>> cd /usr/src/usr.sbin/cron/
>> make obj && make depend && make && make install
>>
>> but i don't know how to enabled debug symbols ..
> 
> Easiest would be to do:
> 
> cd /usr/src/usr.sbin/cron/
> make obj && make depend && make DEBUG=-g && make install
> 
>  - todd
> 

Thank you... I will remember this ..

Re: Cron running at 99% CPU for seemingly no reason

[Hrvoje Popovski]

On 15.5.2022. 15:38, Todd C. Miller wrote:
> On Sun, 15 May 2022 14:29:28 +0200, Hrvoje Popovski wrote:
> 
>> I'm seeing same as Stephan on few servers in lab.
>> I've killed cron and did ktrace -i cron. Is this ok?
>> In attachment you can find kdump -f ktrace.out output.
> 
> That's very odd.  It looks like cron parses root's crontab and then
> somehow gets into a cpu loop.  I don't see how that can happen from
> code inspection.  What would be most useful is to get a stack trace
> of cron when this occurs but that will require rebuilding cron from
> source with debug symbols.

Hi,

I know how to rebuild cron

cd /usr/src/usr.sbin/cron/
make obj && make depend && make && make install

but i don't know how to enabled debug symbols ..

Re: Cron running at 99% CPU for seemingly no reason

[Hrvoje Popovski]

On 15.5.2022. 14:39, Hrvoje Popovski wrote:
> On 15.5.2022. 14:29, Hrvoje Popovski wrote:
>> On 15.5.2022. 12:32, Claudio Jeker wrote:
>>> Also for cron, please attach ktrace to the cron process for a few seconds
>>> and look at the kdump of that. Most probably it is constantly woken up for
>>> some reasons.
>>
>> Hi,
>>
>> I'm seeing same as Stephan on few servers in lab.
>> I've killed cron and did ktrace -i cron. Is this ok?
>> In attachment you can find kdump -f ktrace.out output.
> 
> In attachment you can find kdump from server where cron is ok and from
> server where cron is at 99% ...
> 

it seems that last mail didn't pass so here's kdump output
https://kosjenka.srce.hr/~hrvoje/openbsd/cron-kdump-ok.txt
https://kosjenka.srce.hr/~hrvoje/openbsd/cron-kdump-not-ok.txt

Re: Cron running at 99% CPU for seemingly no reason

[Hrvoje Popovski]

On 15.5.2022. 12:32, Claudio Jeker wrote:
> Also for cron, please attach ktrace to the cron process for a few seconds
> and look at the kdump of that. Most probably it is constantly woken up for
> some reasons.

Hi,

I'm seeing same as Stephan on few servers in lab.
I've killed cron and did ktrace -i cron. Is this ok?
In attachment you can find kdump -f ktrace.out output.

x3550m4# kdump -f ktrace.out
 64517 ktrace   RET   ktrace 0
 64517 ktrace   CALL  
mmap(0,0x4c,0x3,0x1002,-1,0)
 64517 ktrace   RET   mmap 15937734819840/0xe7ecb05c000
 64517 ktrace   CALL  execve(0x7f7dea50,0x7f7df048,0x7f7df058)
 64517 ktrace   NAMI  "/sbin/cron"
 64517 ktrace   RET   execve -1 errno 2 No such file or directory
 64517 ktrace   CALL  execve(0x7f7dea50,0x7f7df048,0x7f7df058)
 64517 ktrace   NAMI  "/usr/sbin/cron"
 64517 ktrace   ARGS
[0] = "cron"
 64517 cron NAMI  "/usr/libexec/ld.so"
 64517 cron RET   execve 0
 64517 cron CALL  getentropy(0x7f7cfdf0,40)
 64517 cron RET   getentropy 0
 64517 cron CALL  getentropy(0x7f7cfdf0,40)
 64517 cron RET   getentropy 0
 64517 cron CALL  
mmap(0,0x4000,0,0x1002,-1,0)
 64517 cron RET   mmap 13386418999296/0xc2cc4bfd000
 64517 cron CALL  mprotect(0xc2cc4bfe000,0x2000,0x3)
 64517 cron RET   mprotect 0
 64517 cron CALL  
mmap(0,0x1000,0x3,0x1002,-1,0)
 64517 cron RET   mmap 13387667914752/0xc2d0f30c000
 64517 cron CALL  issetugid()
 64517 cron RET   issetugid 0
 64517 cron CALL  mprotect(0xc2cb555f000,0x1000,0x1)
 64517 cron RET   mprotect 0
 64517 cron CALL  
mmap(0,0x1000,0x3,0x1002,-1,0)
 64517 cron RET   mmap 13386366095360/0xc2cc1989000
 64517 cron CALL  
mmap(0,0x1000,0x3,0x1002,-1,0)
 64517 cron RET   mmap 13387417739264/0xc2d00476000
 64517 cron CALL  
mmap(0,0x1000,0x3,0x1002,-1,0)
 64517 cron RET   mmap 13387366080512/0xc2cfd332000
 64517 cron CALL  
mmap(0,0x1000,0x3,0x1002,-1,0)
 64517 cron RET   mmap 13384896667648/0xc2c6a02e000
 64517 cron CALL  
mmap(0,0x1000,0x3,0x1002,-1,0)
 64517 cron RET   mmap 13388899835904/0xc2d589e6000
 64517 cron CALL  
mmap(0,0x1000,0x3,0x1002,-1,0)
 64517 cron RET   mmap 13388106907648/0xc2d295b4000
 64517 cron CALL  
mmap(0,0x1000,0x3,0x1002,-1,0)
 64517 cron RET   mmap 13388940087296/0xc2d5b049000
 64517 cron CALL  
mmap(0,0x1000,0x3,0x1002,-1,0)
 64517 cron RET   mmap 13386492366848/0xc2cc91f5000
 64517 cron CALL  
mmap(0,0x1000,0x3,0x1002,-1,0)
 64517 cron RET   mmap 13385890013184/0xc2ca5382000
 64517 cron CALL  
mmap(0,0x1000,0x3,0x1002,-1,0)
 64517 cron RET   mmap 13386167992320/0xc2cb5c9c000
 64517 cron CALL  
mmap(0,0x1000,0x3,0x1002,-1,0)
 64517 cron RET   mmap 13388150337536/0xc2d2bf1f000
 64517 cron CALL  
mmap(0,0x1000,0x3,0x1002,-1,0)
 64517 cron RET   mmap 13388628615168/0xc2d4873e000
 64517 cron CALL  
mmap(0,0x1000,0x3,0x1002,-1,0)
 64517 cron RET   mmap 13388490608640/0xc2d403a1000
 64517 cron CALL  open(0xc2cb535f526,0x1)
 64517 cron NAMI  "/var/run/ld.so.hints"
 64517 cron RET   open 3
 64517 cron CALL  fstat(3,0x7f7cfc20)
 64517 cron STRU  struct stat { dev=1028, ino=5560594, mode=-r--r--r-- , 
nlink=1, uid=0<"root">, gid=0<"wheel">, rdev=22186032, atime=1652606555<"May 15 
11:22:35 2022">.291841835, mtime=1652606555<"May 15 11:22:35 2022">.291841835, 
ctime=1652606555<"May 15 11:22:35 2022">.291841835, size=13459, blocks=28, 
blksize=16384, flags=0x0, gen=0xa9be6c86 }
 64517 cron RET   fstat 0
 64517 cron CALL  mmap(0,0x3493,0x1,0x2,3,0)
 64517 cron RET   mmap 13388976291840/0xc2d5d2d
 64517 cron CALL  
mmap(0,0x1000,0x3,0x1002,-1,0)
 64517 cron RET   mmap 13387006468096/0xc2ce7c3e000
 64517 cron CALL  close(3)
 64517 cron RET   close 0
 64517 cron CALL  open(0xc2d5d2d1dab,0x1)
 64517 cron NAMI  "/usr/lib/libc.so.96.1"
 64517 cron RET   open 3
 64517 cron CALL  fstat(3,0x7f7cfcf0)
 64517 cron STRU  struct stat { dev=1029, ino=181998, mode=-r--r--r-- , 
nlink=1, uid=0<"root">, gid=7<"bin">, rdev=812000, atime=1652606551<"May 15 
11:22:31 2022">.831851254, mtime=1652606551<"May 15 11:22:31 2022">.861850623, 
ctime=1652606553<"May 15 11:22:33 2022">.101848106, size=3618608, blocks=7104, 
blksize=16384, flags=0x0, gen=0x5a285317 }
 64517 cron RET   fstat 0
 64517 cron CALL  read(3,0x7f7cecf0,0x1000)
 64517 cron GIO   fd 3 read 4096 bytes
   
"\^?ELF\^B\^A\^A\0\0\0\0\0\0\0\0\0\^C\0>\0\^A\0\0\w\^C\0\0\0\0\0@\0\0\0\0\0\0\0\M-0+7\0\0\0\0\0\0\0\0\0@\08\0\v\0@\0.\0,\0\^F\0\0\0\^D\0\0\0@\0\0\0\0\0\0\0@\0\0\0\0\0\0\0@\0\0\0\0\0\0\0h\^B\0\0\0\0\0\0h\^B\0\0\0\0\0\0\b\0\0\

\0\0\0\0\0\^A\0\0\0\^D\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0,g\^C\0\0\0\0\0,g\^C\0\0\0\0\0\0\^P\0\0\0\0\0\0\^A\0\0\0\^E\0\0\g\^C\0\0\0\0\w\^C\0\0\0\0\w\^C\0\0\0\0\0\M-PK
\0\0\0\0\0\M-PK

\0\0\0\0\0\0\^P

Re: dmesg - cpu, smt, core, package

[Hrvoje Popovski]

On 10.2.2022. 20:03, Mihai Popescu wrote:
>> you mean gaps because HT is disabled ?
> 
> I think they are disabled from the factory, cores that are not 100%
> functional, i.e defects.
> There is one line for a family, the luckiest ones have the maximum
> number of cores and $$$, the rest are lower but still functional on
> the advertised cores.
> 

if i enable only 4 core per CCD (not sure if this is right term) i'm
getting right numbers

without HT
smc24# dmesg | grep smt
cpu0: smt 0, core 0, package 0
cpu1: smt 0, core 1, package 0
cpu2: smt 0, core 2, package 0
cpu3: smt 0, core 3, package 0
cpu4: smt 0, core 4, package 0
cpu5: smt 0, core 5, package 0
cpu6: smt 0, core 6, package 0
cpu7: smt 0, core 7, package 0
cpu8: smt 0, core 8, package 0
cpu9: smt 0, core 9, package 0
cpu10: smt 0, core 10, package 0
cpu11: smt 0, core 11, package 0
cpu12: smt 0, core 12, package 0
cpu13: smt 0, core 13, package 0
cpu14: smt 0, core 14, package 0
cpu15: smt 0, core 15, package 0

with HT
smc24# dmesg | grep smt
cpu0: smt 0, core 0, package 0
cpu1: smt 0, core 1, package 0
cpu2: smt 0, core 2, package 0
cpu3: smt 0, core 3, package 0
cpu4: smt 0, core 4, package 0
cpu5: smt 0, core 5, package 0
cpu6: smt 0, core 6, package 0
cpu7: smt 0, core 7, package 0
cpu8: smt 0, core 8, package 0
cpu9: smt 0, core 9, package 0
cpu10: smt 0, core 10, package 0
cpu11: smt 0, core 11, package 0
cpu12: smt 0, core 12, package 0
cpu13: smt 0, core 13, package 0
cpu14: smt 0, core 14, package 0
cpu15: smt 0, core 15, package 0
cpu16: smt 1, core 0, package 0
cpu17: smt 1, core 1, package 0
cpu18: smt 1, core 2, package 0
cpu19: smt 1, core 3, package 0
cpu20: smt 1, core 4, package 0
cpu21: smt 1, core 5, package 0
cpu22: smt 1, core 6, package 0
cpu23: smt 1, core 7, package 0
cpu24: smt 1, core 8, package 0
cpu25: smt 1, core 9, package 0
cpu26: smt 1, core 10, package 0
cpu27: smt 1, core 11, package 0
cpu28: smt 1, core 12, package 0
cpu29: smt 1, core 13, package 0
cpu30: smt 1, core 14, package 0
cpu31: smt 1, core 15, package 0

this if funny :)

Re: dmesg - cpu, smt, core, package

[Hrvoje Popovski]

On 10.2.2022. 16:38, Todd C. Miller wrote:
> On Thu, 10 Feb 2022 08:46:37 +, Stuart Henderson wrote:
> 
>> The numbers come from what's reported by the relevant CPUID instruction,
>> the only one actually used by OpenBSD is smt to disable all but one
>> thread in a core, otherwise they're just for information.
>>
>> I'm not sure the reason for the gaps in numbering on some AMDs, but the
>> documentation just talks about IDs and doesn't imply that they have to
>> be contiguous. (https://www.amd.com/system/files/TechDocs/24594.pdf
>> page 629).
> 
> I'd guess that the gaps represent cores present on the chiplet that
> have been disabled.
> 
>  - todd
> 

you mean gaps because HT is disabled ?


this is with HT enabled 

smc24# dmesg | grep smt
cpu0: smt 0, core 0, package 0
cpu1: smt 0, core 1, package 0
cpu2: smt 0, core 2, package 0
cpu3: smt 0, core 3, package 0
cpu4: smt 0, core 4, package 0
cpu5: smt 0, core 5, package 0
cpu6: smt 0, core 8, package 0
cpu7: smt 0, core 9, package 0
cpu8: smt 0, core 10, package 0
cpu9: smt 0, core 11, package 0
cpu10: smt 0, core 12, package 0
cpu11: smt 0, core 13, package 0
cpu12: smt 0, core 16, package 0
cpu13: smt 0, core 17, package 0
cpu14: smt 0, core 18, package 0
cpu15: smt 0, core 19, package 0
cpu16: smt 0, core 20, package 0
cpu17: smt 0, core 21, package 0
cpu18: smt 0, core 24, package 0
cpu19: smt 0, core 25, package 0
cpu20: smt 0, core 26, package 0
cpu21: smt 0, core 27, package 0
cpu22: smt 0, core 28, package 0
cpu23: smt 0, core 29, package 0
cpu24: smt 1, core 0, package 0
cpu25: smt 1, core 1, package 0
cpu26: smt 1, core 2, package 0
cpu27: smt 1, core 3, package 0
cpu28: smt 1, core 4, package 0
cpu29: smt 1, core 5, package 0
cpu30: smt 1, core 8, package 0
cpu31: smt 1, core 9, package 0
cpu32: smt 1, core 10, package 0
cpu33: smt 1, core 11, package 0
cpu34: smt 1, core 12, package 0
cpu35: smt 1, core 13, package 0
cpu36: smt 1, core 16, package 0
cpu37: smt 1, core 17, package 0
cpu38: smt 1, core 18, package 0
cpu39: smt 1, core 19, package 0
cpu40: smt 1, core 20, package 0
cpu41: smt 1, core 21, package 0
cpu42: smt 1, core 24, package 0
cpu43: smt 1, core 25, package 0
cpu44: smt 1, core 26, package 0
cpu45: smt 1, core 27, package 0
cpu46: smt 1, core 28, package 0
cpu47: smt 1, core 29, package 0



dmesg
OpenBSD 7.0-current (GENERIC.MP) #323: Wed Feb  9 21:05:37 MST 2022
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 68497002496 (65323MB)
avail mem = 66403860480 (63327MB)
random: good seed from bootblocks
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 3.3 @ 0xa9d1c000 (71 entries)
bios0: vendor American Megatrends Inc. version "2.3" date 10/20/2021
bios0: Supermicro AS -1114S-WTRT
acpi0 at bios0: ACPI 6.0
acpi0: sleep states S0 S5
acpi0: tables DSDT FACP SSDT SPMI SSDT FIDT MCFG SSDT SSDT BERT HPET
IVRS PCCT SSDT CRAT CDIT SSDT WSMT APIC ERST HEST
acpi0: wakeup devices B000(S3) C000(S3) B010(S3) C010(S3) B030(S3)
C030(S3) B020(S3) C020(S3) B100(S3) C100(S3) B110(S3) C110(S3) B130(S3)
C130(S3) B120(S3) C120(S3)
acpitimer0 at acpi0: 3579545 Hz, 32 bits
acpimcfg0 at acpi0
acpimcfg0: addr 0xe000, bus 0-255
acpihpet0 at acpi0: 14318180 Hz
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: AMD EPYC 7413 24-Core Processor, 2650.38 MHz, 19-01-01
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,PCID,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,INVPCID,PQM,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,SHA,UMIP,PKU,IBPB,IBRS,STIBP,SSBD,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu0: 32KB 64b/line 8-way I-cache, 32KB 64b/line 8-way D-cache, 512KB
64b/line 8-way L2 cache
cpu0: ITLB 64 4KB entries fully associative, 64 4MB entries fully
associative
cpu0: DTLB 64 4KB entries fully associative, 64 4MB entries fully
associative
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=1.1, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: AMD EPYC 7413 24-Core Processor, 2650.00 MHz, 19-01-01
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,PCID,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,INVPCID,PQM,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,SHA,UMIP,PKU,IBPB,IBRS,STIBP,SSBD,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu1: 32KB 64b/line 8-way I-cache, 32KB 64b/line 8-way D-cache, 512KB
64b/line 8-way 

Re: dmesg - cpu, smt, core, package

[Hrvoje Popovski]

On 9.2.2022. 19:04, Kapetanakis Giannis wrote:
> On 09/02/2022 19:48, Mihai Popescu wrote:
>> $ dmesg | grep smt
>> cpu0: smt 0, core 0, package 0
>> cpu1: smt 1, core 0, package 0
>> cpu2: smt 0, core 1, package 0
>> cpu3: smt 1, core 1, package 0
>>
>> for
>>
>> AMD A8-5500B APU with Radeon(tm) HD Graphics, 3194.46 MHz, 15-10-01
>>
>> What could be the "smt" thing?
>>
> 
> multi threading
> 
> # sysctl hw.smt
> 
> G
> 

and i think that "package" is cpu socket ..

dmesg - cpu, smt, core, package

[Hrvoje Popovski]

Hi all,

in one supermicro box in dmesg i'm seeing this

smc24# dmesg | grep smt
cpu0: smt 0, core 0, package 0
cpu1: smt 0, core 1, package 0
cpu2: smt 0, core 2, package 0
cpu3: smt 0, core 3, package 0
cpu4: smt 0, core 4, package 0
cpu5: smt 0, core 5, package 0
cpu6: smt 0, core 8, package 0
cpu7: smt 0, core 9, package 0
cpu8: smt 0, core 10, package 0
cpu9: smt 0, core 11, package 0
cpu10: smt 0, core 12, package 0
cpu11: smt 0, core 13, package 0
cpu12: smt 0, core 16, package 0
cpu13: smt 0, core 17, package 0
cpu14: smt 0, core 18, package 0
cpu15: smt 0, core 19, package 0
cpu16: smt 0, core 20, package 0
cpu17: smt 0, core 21, package 0
cpu18: smt 0, core 24, package 0
cpu19: smt 0, core 25, package 0
cpu20: smt 0, core 26, package 0
cpu21: smt 0, core 27, package 0
cpu22: smt 0, core 28, package 0
cpu23: smt 0, core 29, package 0

should core be identical to cpu number?


this is from dell r7515

r7515# dmesg | grep smt
cpu0: smt 0, core 0, package 0
cpu1: smt 0, core 1, package 0
cpu2: smt 0, core 2, package 0
cpu3: smt 0, core 3, package 0
cpu4: smt 0, core 4, package 0
cpu5: smt 0, core 5, package 0
cpu6: smt 0, core 6, package 0
cpu7: smt 0, core 7, package 0
cpu8: smt 0, core 8, package 0
cpu9: smt 0, core 9, package 0
cpu10: smt 0, core 10, package 0
cpu11: smt 0, core 11, package 0
cpu12: smt 0, core 12, package 0
cpu13: smt 0, core 13, package 0
cpu14: smt 0, core 14, package 0
cpu15: smt 0, core 15, package 0
cpu16: smt 1, core 0, package 0
cpu17: smt 1, core 1, package 0
cpu18: smt 1, core 2, package 0
cpu19: smt 1, core 3, package 0
cpu20: smt 1, core 4, package 0
cpu21: smt 1, core 5, package 0
cpu22: smt 1, core 6, package 0
cpu23: smt 1, core 7, package 0
cpu24: smt 1, core 8, package 0
cpu25: smt 1, core 9, package 0
cpu26: smt 1, core 10, package 0
cpu27: smt 1, core 11, package 0
cpu28: smt 1, core 12, package 0
cpu29: smt 1, core 13, package 0
cpu30: smt 1, core 14, package 0
cpu31: smt 1, core 15, package 0

Re: apu2e4 intermittent network freeze

[Hrvoje Popovski]

On 31.1.2022. 17:03, Amarendra Godbole wrote:
> [...]
> 
> Thanks for your response(s). A few releases ago I did have a bridge,
> but realized it causes an overall throughput drop rather than using
> individual interfaces directly. I should have clarified -- even though
> both interfaces are on the same subnet, only one is connected at any
> given time, until yesterday, when I started seeing the issue on em1.
> 
> Let me give a try to veb(4) and vport(4).
> 
> -Amarendra


It would be great that em(4) have multiqueue support, that box with
veb(4) and "parallel forwarding" diff on tech@ would kick ass :)

Re: apu2e4 intermittent network freeze

[Hrvoje Popovski]

On 31.1.2022. 13:44, Łukasz Moskała wrote:
> W dniu 31.01.2022 o 02:44, Amarendra Godbole pisze:
>> My home network has a PC Engines apu2e4 running OpenBSD 7.0, acting as
>> a firewall/router, dhcp server, and DNS server. A Ruckus wifi AP
>> receives a fixed DHCP address from apu2e4. All devices connect to the
>> AP, and receive IP address in the same subnet. apu2e4 has em0, em1 and
>> em2, of which em0 is uplink from Comcast, and em1 and em2 are fixed to
>> 192.168.10.1 and .2 respectively. I have dhcpd and unbound listening
>> on both em1 and em2.
>>
>> Normally my laptop that receives an IP of 192.168.10.105 is able to
>> ping the ap2e4 at 192.168.10.1 (and even ssh into it). Today I lost
>> that connectivity first, and ping stopped working. A restart of
>> network on apu2e4 got it working again. The problem kept repeating
>> every few minutes (maybe 5 or so?), till I restarted network on the
>> apu2e4/OpenBSD host.
>>
>> What changed today? In the morning, I applied the last two patches 009
>> (expat) and 010 (vmm). So I uninstalled those, but as guessed, the
>> problem did not go away. So now I switched to the other channel (em1),
>> and the connectivity has been stable so far.
>>
>> I am completely in the dark here and do not have a clue as to what may
>> have happened - something to do with networking, and possibly an
>> ethernet channel going bad on apu2e4 since the second one works? Can
>> anyone provide a few pointers on where I should start looking?
>>
>> Thanks in advance. dmesg attached.
>>
>> -Amarendra
> 
> So, you have em1 with 192.168.10.1/24 and em2 with 192.168.10.2/24?
> 
> Having two interfaces in the same subnet is a bad idea (unless they are
> in seperate routing domains)
> 
> I think that what you want to do is:
>  - create bridge0

you mean veb(4)? right? :)

>  - move 192.168.10.1/24 address to bridge0

and vport(4) :)



>  - remove IP address from em1 and em2
>  - attach em1 and em2 as bridge0 members
>  - make dhcpd, unbound and whatever listen on bridge0
> 
> Alternatively, change em2 IP address to be in other subnet than em1, for
> example 192.168.20.1/24
> 

Re: CPU recommendation

[Hrvoje Popovski]

On 29.11.2021. 15:55, Barbaros Bilek wrote:
> Hello @misc,
> 
> I’m network administrator at a Hotel. We have nearly ~=1600 users
> concurrently.
> I’m trying to figure out which hardware covers my pc based OpenBSD firewall.
> Disk : 1 TB SSD
> RAM : 16 GB
> Ethernet : Intel i211AT
> But what about CPU. As far as I know CPU frequency is more important at
> OpenBSD cause there is netlock() etc.
> Right?
> 
> So which CPU is better at the moment?
> Intel Core i3-6320 @ 3.90GHz
> Intel Core i7-7700 @ 3.60GHz
> 

Hi,

I would go with at least 4 cores (8 cores are better) without HT and
with X540-T/X550-T ix(4) interface rather that em(4), even on 1G...
that's because openbsd doesn't have multiqueue support for em(4) yet,
but for ix(4) it does.

if you can wait for em(4) to gain multiqueue support, go with em ...but
it seems to me that motivation to have mq em(4) is not that high :)

openbsd could be unlocked soon and multiqueue support of network cards
is fundamental for that to happen and in that moment you would like to
have a reasonable amount of core and proper mq interface

Re: rpki-client and BLACKHOLE routes

[Hrvoje Popovski]

On 23.6.2021. 12:09, Claudio Jeker wrote:
> On Wed, Jun 23, 2021 at 11:40:25AM +0200, Hrvoje Popovski wrote:
>> Hi all,
>>
>> fist of all, thank you for rpki-client, it's so easy to use it and to
>> get the job done.
>> I'm playing with rpki-client and denying ovs invalid statement and I've
>> seen that with default ovs config statement (deny from ebgp ovs invalid)
>> BLACKHOLE routes are blocked/invalid.
>>
>> What is the right way to allow BLACKHOLE routes through rpki ? Or if
>> someone can give me a hint on what to do.
>>
> 
> BLACKHOLE routes normally have a more specific check so you can re-allow
> them back after the ovs invalid check (for that you need to take away the
> quick from the default ruleset or actually allow quick the blackholes
> before).
> 
> I guess you can use something along the lines of:
> allow quick from group clients inet prefixlen 32 community $BLACKHOLE set 
> nexthop blackhole
> allow quick from group clients inet6 prefixlen 128 community $BLACKHOLE set 
> nexthop blackhole
> 
> I guess you also have some client prefix-sets that should be added to the
> filter rule so that one client can not blackhole for another.
> 
> BLACKHOLE routes are done in many ways and I'm not sure if there is
> consensus who is allowed to announce what. Also if there are multiple
> paths to the destination should the blackhole only be active if the
> covering route is from the same peer?
> 

Thank you guys for rpki-client. Now we have block invalids in cix and
blackhole routes still works :)

Thank you ..

Re: Exoscale VPS panic on boot, 10-25 snapshot

[Hrvoje Popovski]

On 26.10.2021. 1:16, Ashlen wrote:
> Here is as much information as I could get. After upgrading to a
> snapshot earlier today (October 25th), the Exoscale VPS panics on boot.
> I use this VPS to self-host synapse (a Matrix homeserver, for
> messaging).
> 
> I can't copy and paste from the web console that Exoscale provides so I
> had to transcribe all of it by hand, I hope it's all accurate. In
> particular, the correct number of spaces between fields in `ps` output
> is unknown to me, but everything else should be OK I think. I tried to
> get a traceback for cpu1 as well, but the console hangs when I issue
> `machine ddbcpu 1` so only the traceback for cpu0 was available to me.
> 
> Booting /bsd.sp instead of /bsd.mp or scaling the VPS down to one core
> appears to make no difference, the panic happens regardless.
> 
> Any suggestions to get my VPS back up and running would be much
> appreciated, I feel pretty lost with what to do next. Thanks.
> 

could you try lastes snapshot with sysupgrade? i had same problem on
Dell r620 and latest snapshot fix that panic ..

Re: ipsec with default route and routing of internal networks

[Hrvoje Popovski]

On 14.9.2021. 13:12, Hrvoje Popovski wrote:
> On 13.9.2021. 15:52, Stuart Henderson wrote:
>> On 2021-09-13, Hrvoje Popovski  wrote:
>>> On 13.9.2021. 14:08, Tom Smyth wrote:
>>>> Can you do  an exception for the ranges ...  so internet - private ips
>>>> you dont want over the tunnel)
>>>>
>>>> ike esp from 10.90.0.0/24 <http://10.90.0.0/24> to any encrypt  
>>>> and 
>>>>
>>>>  10.90.0.0/24 <http://10.90.0.0/24> to   NOT  [networks you dont want
>>>> over the tunnel)  ? 
>>>>
>>>
>>> :) this was the first thought that i've had ... but i couldn't find how
>>> to do it ... at least in man ipsec.conf or isakmpd.conf
>>>
>>>
>>
>> You do this with a "bypass flow" in /etc/ipsec.conf:
>>
>> flow from $network/$prefix to $network/$prefix type bypass
>>
>> and loading it with ipsecctl. Note if you use iked, you cannot configure
>> this directly in iked.conf, but you can still use ipsecctl and ipsec.conf
>> for this purpose in conjunction with iked for tunnel setup.
>>
>>
> 
> Thank you guys ... with "type bypass" everything is working as expected
> 
> c/p from config
> ike esp from 10.90.0.0/24 to any \
> local $localip peer $peerip \
> main auth hmac-sha1 enc aes group modp1024 \
> quick enc aes-128-gcm group modp1024 \
> psk 123
> flow from 10.90.0.0/24 to 10.90.0.0/24 type bypass
> flow from 10.90.0.0/24 to 10.91.0.0/24 type bypass
> flow from 10.90.0.0/24 to 10.92.0.0/24 type bypass
> 

and if you have carp (multicast) than you need
flow from 10.90.0.0/24 to 224.0.0.18/32 type bypass

Re: ipsec with default route and routing of internal networks

[Hrvoje Popovski]

On 13.9.2021. 15:52, Stuart Henderson wrote:
> On 2021-09-13, Hrvoje Popovski  wrote:
>> On 13.9.2021. 14:08, Tom Smyth wrote:
>>> Can you do  an exception for the ranges ...  so internet - private ips
>>> you dont want over the tunnel)
>>>
>>> ike esp from 10.90.0.0/24 <http://10.90.0.0/24> to any encrypt  
>>> and 
>>>
>>>  10.90.0.0/24 <http://10.90.0.0/24> to   NOT  [networks you dont want
>>> over the tunnel)  ? 
>>>
>>
>> :) this was the first thought that i've had ... but i couldn't find how
>> to do it ... at least in man ipsec.conf or isakmpd.conf
>>
>>
> 
> You do this with a "bypass flow" in /etc/ipsec.conf:
> 
> flow from $network/$prefix to $network/$prefix type bypass
> 
> and loading it with ipsecctl. Note if you use iked, you cannot configure
> this directly in iked.conf, but you can still use ipsecctl and ipsec.conf
> for this purpose in conjunction with iked for tunnel setup.
> 
> 

Thank you guys ... with "type bypass" everything is working as expected

c/p from config
ike esp from 10.90.0.0/24 to any \
local $localip peer $peerip \
main auth hmac-sha1 enc aes group modp1024 \
quick enc aes-128-gcm group modp1024 \
psk 123
flow from 10.90.0.0/24 to 10.90.0.0/24 type bypass
flow from 10.90.0.0/24 to 10.91.0.0/24 type bypass
flow from 10.90.0.0/24 to 10.92.0.0/24 type bypass




ipsecctl -sa | grep 10.9
flow esp in from 0.0.0.0/0 to 10.90.0.0/24 peer $peerip srcid $localip
dstid $peerip type require
flow esp in from 10.90.0.0/24 to 10.90.0.0/24 type bypass
flow esp in from 10.91.0.0/24 to 10.90.0.0/24 type bypass
flow esp in from 10.92.0.0/24 to 10.90.0.0/24 type bypass

flow esp out from 10.90.0.0/24 to 0.0.0.0/0 peer $peerip srcid $localip
dstid $peerip type require
flow esp out from 10.90.0.0/24 to 10.90.0.0/24 type bypass
flow esp out from 10.90.0.0/24 to 10.91.0.0/24 type bypass
flow esp out from 10.90.0.0/24 to 10.92.0.0/24 type bypass

Re: ipsec with default route and routing of internal networks

[Hrvoje Popovski]

On 13.9.2021. 14:08, Tom Smyth wrote:
> Can you do  an exception for the ranges ...  so internet - private ips
> you dont want over the tunnel)
> 
> ike esp from 10.90.0.0/24  to any encrypt  
> and 
> 
>  10.90.0.0/24  to   NOT  [networks you dont want
> over the tunnel)  ? 
> 

:) this was the first thought that i've had ... but i couldn't find how
to do it ... at least in man ipsec.conf or isakmpd.conf

Re: ipsec with default route and routing of internal networks

[Hrvoje Popovski]

Hi,

On 13.9.2021. 12:58, Tom Smyth wrote:
> Hi Hrvoje, 
> 
> is 10.90.0.0/24  local to your firewall, and if I
> understand your rule,
> ike esp from 10.90.0.0/24  to any    you are saying  
> encrypt all traffic comming from 10.90.0.0/24  
> 
> should the tunnel be more specific ? like 
> 
> from 10.90.0.0/24   to another network across the
> tunnel  
> 

10.90/24 is my local internal network, as other networks (10.91/24,
10.92/24).
i need "ike esp from 10.90.0.0/24 to any"... because hosts on that
network need to go out to internet over ipsec tunnel ... but at the same
time hosts in that 10.90/24 network needs to communicate to other
internal networks...

ipsec with default route and routing of internal networks

[Hrvoje Popovski]

Hi all,

I have a firewall that routes few internal networks, 10.90/24, 10.91/24,
10.92/24. And i have some static routes to other firewalls, but i don't
think that is relevant to this problem.

For network 10.90/24 i have ipsec tunnel, and i need to push any traffic
from that network to the internet, but not to local networks,
over that ipsec tunnel.

something like this:
ike esp from 10.90.0.0/24 to any

I thought that the routing table will take care of that, but i seems
that when ipsec tunnel is up, i can't connect from local networks
(10.91/24, 10.92/24) to 10.90/24 and I can't even ping hosts on the
10.90/24 network ...
something like this ping -I 10.90.0.1 10.90.0.8 ...
traffic from 10.90/24 to the internet is working just fine ..

I need to make network 10.90/24 reachable to all local networks.
Could someone please point me in the right direction on what to look and
configure?

Thank you ..

supermicro bmc and openbsd efi install

[Hrvoje Popovski]

Hi all,

In supermicro server i only have one m2 nvme disk. Because of that i
need to enable efi boot to make that disk bootable ...
I can mount install.img over bmc as HD image, but boot from that
"virtual disk" won't start...

is there any way to install openbsd efi image on supermicro server over
their bmc ?

In legacy mode openbsd installs just fine but i can't make it boot in
bios ...

Thanks ...

Re: Resolved - Was: Performance tuning PF.

[Hrvoje Popovski]

On 27.7.2021. 17:36, Christopher Sean Hilton wrote:
> On Sat, Jul 24, 2021 at 10:24:28AM -, Stuart Henderson wrote:
>> On 2021-07-23, Christopher Sean Hilton  wrote:
>>> On Fri, Jul 23, 2021 at 11:19:35AM -0400, Chris Hilton wrote:
> 
> [ ...snip... ]
> 
>>>
>>> Answering my own question, it looks like the Xeon D is intels newest
>>> low power stuff. I'll look there.
>>
>> Not particularly new, Xeon D 1500 series are from 2016 or so and still
>> seem to be the range to go for if you care about good power use. Look
>> at supermicro X10SDV (Xeon D 1500 series) or M11SDV (AMD EPYC). Sadly
>> the M11SDV only has copper nics, X10SDV have decent ix(4) SFP+ plus
>> some copper. (X10 is an older supermicro range, I'm not sure what the
>> availability is like).
>>
>> supermicro, if you're reading, an EPYC board with a couple of SFP28
>> onboard would be nice...
>>
>> Sample dmesg from one of the X10SDV models - em and ix are onboard,
>> ixl is a card:
>>
>> OpenBSD 6.8-current (GENERIC.MP) #220: Thu Dec 10 20:03:29 MST 2020
>> dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
> 
> [ ...snip ]
> 
> Thanks to everyone for the answers that they provided. Just a late
> followup here. I thought through my testing rig and realized that it
> was slightly flawed. I was originally using one of the Atoms as an
> iperf endpoint. That obviously messed up the tests. I retested using a
> pair of machine which I know can saturate a 1Gb/s connection. My
> new test rig is a pair of MacBook Pro's with Thunderbolt Ethernet
> adapters:
> 
> * With just a GigE switch connecting the test machines, I measured a
>   transfer rate of 942 Mb/s. The test program was iperf3.
> 
> * With OpenBSD 6.8 running a bridged configuration on an Intel Atom
>   D525 with internal and external "em" nics, and filtering using pf.
>   I measured a rate of 775 ~ 850 Mb/s. Again, the test program was
>   iperf3.
> 


maybe you can update to snapshot or 6.9 and try veb(4) instead of
bridge(4) ?


> Testing the routed configuration on my Atom C2758 is a little more
> difficult. I'll set that up next week. I expect that the transfer rate
> through that combination will be a little lower since routing is more
> difficult than bridging.



> 
> I am currently shopping Intel Xeon-D hardware. I plan to eventually
> replace the D525 bridge with the C2758 running in a bridged
> configuration and use new Xeon-D hardware for the router.
> 
> -- Chris
> 
> 

Re: OpenBSD 6.9 on Hetzner cloud server

[Hrvoje Popovski]

On 22.7.2021. 16:33, Matthias Schmidt wrote:
> Hi,
> 
> * Hrvoje Popovski wrote:
>> Hi all,
>>
>> I'm thinking of getting Hetzner cloud server and install OpenBSD stable
>> on it...
>>
>> Does anyone have experience with it? Is it complicated to install
>> OpenBSD on it? And of course, is it stable?
> 
> I ran OpenBSD -stable on a Hetzner cloud server (previously known as
> VPS) for several years and it was rock stable and easy to install.  I
> moved away from Hetzner since their IP space has a bad reputation and is
> often on deny-lists by default.  Even if your server is well maintained
> you end up on such a list as collateral damage.
> 
> Nowadays, I have all my servers with IONOS.  Their IP space has better
> reputation and their VPS product is much much faster (it's based on
> VMWare and not on KVM).  OpenBSD easy is simple since you can upload
> custom ISOs and have remote console access.
> 
> Cheers
> 
>   Matthias
> 


Thank you guys for information ..

OpenBSD 6.9 on Hetzner cloud server

[Hrvoje Popovski]

Hi all,

I'm thinking of getting Hetzner cloud server and install OpenBSD stable
on it...

Does anyone have experience with it? Is it complicated to install
OpenBSD on it? And of course, is it stable?

Thank you

Re: rpki-client and BLACKHOLE routes

[Hrvoje Popovski]

On 23.6.2021. 12:09, Claudio Jeker wrote:
> On Wed, Jun 23, 2021 at 11:40:25AM +0200, Hrvoje Popovski wrote:
>> Hi all,
>>
>> fist of all, thank you for rpki-client, it's so easy to use it and to
>> get the job done.
>> I'm playing with rpki-client and denying ovs invalid statement and I've
>> seen that with default ovs config statement (deny from ebgp ovs invalid)
>> BLACKHOLE routes are blocked/invalid.
>>
>> What is the right way to allow BLACKHOLE routes through rpki ? Or if
>> someone can give me a hint on what to do.
>>
> 
> BLACKHOLE routes normally have a more specific check so you can re-allow
> them back after the ovs invalid check (for that you need to take away the
> quick from the default ruleset or actually allow quick the blackholes
> before).
> 
> I guess you can use something along the lines of:
> allow quick from group clients inet prefixlen 32 community $BLACKHOLE set 
> nexthop blackhole
> allow quick from group clients inet6 prefixlen 128 community $BLACKHOLE set 
> nexthop blackhole
> 
> I guess you also have some client prefix-sets that should be added to the
> filter rule so that one client can not blackhole for another.
> 
> BLACKHOLE routes are done in many ways and I'm not sure if there is
> consensus who is allowed to announce what. Also if there are multiple
> paths to the destination should the blackhole only be active if the
> covering route is from the same peer?


This is exactly what i need, thank you ...

rpki-client and BLACKHOLE routes

[Hrvoje Popovski]

Hi all,

fist of all, thank you for rpki-client, it's so easy to use it and to
get the job done.
I'm playing with rpki-client and denying ovs invalid statement and I've
seen that with default ovs config statement (deny from ebgp ovs invalid)
BLACKHOLE routes are blocked/invalid.

What is the right way to allow BLACKHOLE routes through rpki ? Or if
someone can give me a hint on what to do.

Thank you...

Re: gnome, gdm problem on lenovo e14 gen2

[Hrvoje Popovski]

On 4.5.2021. 13:58, Nam Nguyen wrote:
> Hrvoje Popovski writes:
> 
>> Problem is that when i should get login screen, gdm to ask me for user
>> and password, i'm getting blank grey screen ..
>>
>> after moving through terminals with ctrl-alt fX, from time to time i can
>> get this (screenshot below)
>> https://kosjenka.srce.hr/~hrvoje/openbsd/gdm1.jpg
>> https://kosjenka.srce.hr/~hrvoje/openbsd/gdm2.jpg
>>
>> in both cases, i can't click on anything in login screen ..
>>
>> I'm not much of a desktop user and if someone have clue what i'm doing
>> wrong please tell me :)
> 
> Thanks for reporting this. I also get this with my radeon 6850 where the
> screen is grey. If I switch back and forth through terminals I might
> eventually get the screen to render. Nothing is clickable.
> 
> In contrast gnome works on my thinkpad x230i, which uses intel(4).
> 

Yeah, I've tried whatever I knew or found on the net, but it seems to me
that gnome or gdm or something, just doesn't work on my laptop

OpenBSD 6.9-current (GENERIC.MP) #1: Wed May  5 18:44:19 CEST 2021
hrv...@e14gen2.srce.hr:/sys/arch/amd64/compile/GENERIC.MP
real mem = 7742496768 (7383MB)
avail mem = 7492403200 (7145MB)
random: good seed from bootblocks
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 3.2 @ 0xbf913000 (62 entries)
bios0: vendor LENOVO version "R1AET36W (1.12 )" date 03/15/2021
bios0: LENOVO 20T6000TSC
acpi0 at bios0: ACPI 6.3
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP SSDT SSDT IVRS SSDT SSDT TPM2 SSDT MSDM BATB
HPET APIC MCFG SBST WSMT VFCT SSDT CRAT CDIT FPDT SSDT SSDT SSDT BGRT
UEFI SSDT SSDT
acpi0: wakeup devices GPP3(S3) GPP4(S4) GPP5(S3) XHC0(S3) XHC1(S3)
GP19(S3) LID_(S4) SLPB(S3)
acpitimer0 at acpi0: 3579545 Hz, 32 bits
acpihpet0 at acpi0: 14318180 Hz
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: AMD Ryzen 5 4500U with Radeon Graphics, 2370.83 MHz, 17-60-01
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,PQM,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,SHA,UMIP,IBPB,IBRS,STIBP,SSBD,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu0: 32KB 64b/line 8-way I-cache, 32KB 64b/line 8-way D-cache, 512KB
64b/line 8-way L2 cache
cpu0: ITLB 64 4KB entries fully associative, 64 4MB entries fully
associative
cpu0: DTLB 64 4KB entries fully associative, 64 4MB entries fully
associative
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=1.1, IBE
cpu1 at mainbus0: apid 1 (application processor)
cpu1: AMD Ryzen 5 4500U with Radeon Graphics, 2370.56 MHz, 17-60-01
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,PQM,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,SHA,UMIP,IBPB,IBRS,STIBP,SSBD,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu1: 32KB 64b/line 8-way I-cache, 32KB 64b/line 8-way D-cache, 512KB
64b/line 8-way L2 cache
cpu1: ITLB 64 4KB entries fully associative, 64 4MB entries fully
associative
cpu1: DTLB 64 4KB entries fully associative, 64 4MB entries fully
associative
cpu1: disabling user TSC (skew=-576239375)
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 2 (application processor)
cpu2: AMD Ryzen 5 4500U with Radeon Graphics, 2370.56 MHz, 17-60-01
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,PQM,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,SHA,UMIP,IBPB,IBRS,STIBP,SSBD,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu2: 32KB 64b/line 8-way I-cache, 32KB 64b/line 8-way D-cache, 512KB
64b/line 8-way L2 cache
cpu2: ITLB 64 4KB entries fully associative, 64 4MB entries fully
associative
cpu2: DTLB 64 4KB entries fully associative, 64 4MB entries fully
associative
cpu2: disabling user TSC (skew=-576239362)
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 4 (application processor)
cpu3: AMD Ryzen 5 4500U with Radeon Graphics, 2370.57 MHz, 17-60-01
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE

gnome, gdm problem on lenovo e14 gen2

[Hrvoje Popovski]

Hi all,

I've installed a snapshot on e14gen2 and the installation went smooth.
Gnome was installed and configured based on
/usr/local/share/doc/pkg-readmes/gnome.
Problem is that when i should get login screen, gdm to ask me for user
and password, i'm getting blank grey screen ..

after moving through terminals with ctrl-alt fX, from time to time i can
get this (screenshot below)
https://kosjenka.srce.hr/~hrvoje/openbsd/gdm1.jpg
https://kosjenka.srce.hr/~hrvoje/openbsd/gdm2.jpg

in both cases, i can't click on anything in login screen ..

I'm not much of a desktop user and if someone have clue what i'm doing
wrong please tell me :)

Tnx ..



cat /etc/rc.conf.local
multicast=YES
pkg_scripts=messagebus avahi_daemon gdm


OpenBSD 6.9-current (GENERIC.MP) #0: Sun May  2 23:36:18 MDT 2021
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 7742496768 (7383MB)
avail mem = 7492407296 (7145MB)
random: good seed from bootblocks
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 3.2 @ 0xbf913000 (62 entries)
bios0: vendor LENOVO version "R1AET36W (1.12 )" date 03/15/2021
bios0: LENOVO 20T6000TSC
acpi0 at bios0: ACPI 6.3
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP SSDT SSDT IVRS SSDT SSDT TPM2 SSDT MSDM BATB
HPET APIC MCFG SBST WSMT VFCT SSDT CRAT CDIT FPDT SSDT SSDT SSDT BGRT
UEFI SSDT SSDT
acpi0: wakeup devices GPP3(S3) GPP4(S4) GPP5(S3) XHC0(S3) XHC1(S3)
GP19(S3) LID_(S4) SLPB(S3)
acpitimer0 at acpi0: 3579545 Hz, 32 bits
acpihpet0 at acpi0: 14318180 Hz
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: AMD Ryzen 5 4500U with Radeon Graphics, 2370.85 MHz, 17-60-01
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,PQM,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,SHA,UMIP,IBPB,IBRS,STIBP,SSBD,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu0: 32KB 64b/line 8-way I-cache, 32KB 64b/line 8-way D-cache, 512KB
64b/line 8-way L2 cache
cpu0: ITLB 64 4KB entries fully associative, 64 4MB entries fully
associative
cpu0: DTLB 64 4KB entries fully associative, 64 4MB entries fully
associative
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=1.1, IBE
cpu1 at mainbus0: apid 1 (application processor)
cpu1: AMD Ryzen 5 4500U with Radeon Graphics, 2370.56 MHz, 17-60-01
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,PQM,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,SHA,UMIP,IBPB,IBRS,STIBP,SSBD,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu1: 32KB 64b/line 8-way I-cache, 32KB 64b/line 8-way D-cache, 512KB
64b/line 8-way L2 cache
cpu1: ITLB 64 4KB entries fully associative, 64 4MB entries fully
associative
cpu1: DTLB 64 4KB entries fully associative, 64 4MB entries fully
associative
cpu1: disabling user TSC (skew=-575919403)
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 2 (application processor)
cpu2: AMD Ryzen 5 4500U with Radeon Graphics, 2370.55 MHz, 17-60-01
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,PQM,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,SHA,UMIP,IBPB,IBRS,STIBP,SSBD,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu2: 32KB 64b/line 8-way I-cache, 32KB 64b/line 8-way D-cache, 512KB
64b/line 8-way L2 cache
cpu2: ITLB 64 4KB entries fully associative, 64 4MB entries fully
associative
cpu2: DTLB 64 4KB entries fully associative, 64 4MB entries fully
associative
cpu2: disabling user TSC (skew=-575919378)
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 4 (application processor)
cpu3: AMD Ryzen 5 4500U with Radeon Graphics, 2370.56 MHz, 17-60-01
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,PQM,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,SHA,UMIP,IBPB,IBRS,STIBP,SSBD,XSAVEOPT,XSAVEC,XGETBV1,XSA

Re: OpenBSD on Dell PE R6515

[Hrvoje Popovski]

On 12.4.2021. 20:04, Joerg Streckfuss wrote:
> 
> Hello folks,
> 
> in the past we used Dell servers like PE 1850, PE 2850, PE R730 and PE
> R740. We had good experiences running Openbsd on these systems. These
> models are all Intel based but for another project i'm considering
> giving AMD a chance.
> 
> I'm very interested in the Dell PE R6515 with AMD EPYC 7302P 3GHz,
> 16C/32T CPU and with a mix of NICs (Intel XXV710 10/25 GbE SFP28,
> Broadcom 57416 Dual Port 10 GbE SFP+, Intel i350 Quad Port 1GbE BASE-T).
> 
> The purpose is a Mix of PF firewall and bgp router. In the first stage
> of expansion, the system should be able to handle 10Gbits of traffic.
> Possibly more later.
> 
> Does anyone have experience running OpenBSD on this platform?
> 
> Thanks in advance for feedback,
> 
> Joerg
> 

Hi,

i have r7515 with 7702p which is the same generation as 7302p and it's
working without any problems.

reagring nic card, i would go with connect-x 4 lx for 10/25G, x520 or
x710 for 10G only, and as you mentioned i350 for 1G ...
for broadcom card, i'm not sure ...




dmesg:

r7515# dmesg
OpenBSD 6.9 (GENERIC.MP) #453: Sun Apr  4 19:37:01 MDT 2021
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 549314162688 (523866MB)
avail mem = 532650860544 (507975MB)
random: good seed from bootblocks
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 3.3 @ 0x697a5000 (72 entries)
bios0: vendor Dell Inc. version "2.0.3" date 01/15/2021
bios0: Dell Inc. PowerEdge R7515
acpi0 at bios0: ACPI 6.0
acpi0: sleep states S0 S5
acpi0: tables DSDT FACP BERT HEST HPET APIC MCFG WSMT SLIC SSDT SSDT
EINJ SSDT CRAT CDIT IVRS SSDT
acpi0: wakeup devices PC00(S5) XHCI(S3) PC01(S5) XHCI(S3) PC02(S5)
XHCI(S3) PC03(S5) XHCI(S3)
acpitimer0 at acpi0: 3579545 Hz, 32 bits
acpihpet0 at acpi0: 14318180 Hz
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
ioapic0 at mainbus0: apid 240 pa 0xfec0, version 21, 24 pins, can't
remap
ioapic1 at mainbus0: apid 241 pa 0xe010, version 21, 32 pins, can't
remap
ioapic2 at mainbus0: apid 242 pa 0xc510, version 21, 32 pins, can't
remap
ioapic3 at mainbus0: apid 243 pa 0xaa10, version 21, 32 pins, can't
remap
ioapic4 at mainbus0: apid 244 pa 0xfd10, version 21, 32 pins, can't
remap
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: AMD EPYC 7702P 64-Core Processor, 1996.51 MHz, 17-31-00
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,PQM,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,SHA,UMIP,IBPB,IBRS,STIBP,SSBD,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu0: 32KB 64b/line 8-way I-cache, 32KB 64b/line 8-way D-cache, 512KB
64b/line 8-way L2 cache
cpu0: ITLB 64 4KB entries fully associative, 64 4MB entries fully
associative
cpu0: DTLB 64 4KB entries fully associative, 64 4MB entries fully
associative
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=1.1, IBE
cpu1 at mainbus0: apid 1 (application processor)
cpu1: AMD EPYC 7702P 64-Core Processor, 1996.26 MHz, 17-31-00
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,PQM,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,SHA,UMIP,IBPB,IBRS,STIBP,SSBD,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu1: 32KB 64b/line 8-way I-cache, 32KB 64b/line 8-way D-cache, 512KB
64b/line 8-way L2 cache
cpu1: ITLB 64 4KB entries fully associative, 64 4MB entries fully
associative
cpu1: DTLB 64 4KB entries fully associative, 64 4MB entries fully
associative
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 2 (application processor)
cpu2: AMD EPYC 7702P 64-Core Processor, 1996.26 MHz, 17-31-00
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,PCLMUL,MWAIT,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,SKINIT,TCE,TOPEXT,CPCTR,DBKP,PCTRL3,MWAITX,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,PQM,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,SHA,UMIP,IBPB,IBRS,STIBP,SSBD,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu2: 32KB 64b/line 8-way I-cache, 32KB 64b/line 8-way D-cache, 512KB
64b/line 8-way L2 cache
cpu2: ITLB 64 4KB entries fully associative, 64 4MB entries fully
associative
cpu2: DTLB 64 4KB entries fully associative, 64 4MB entries fully
as

Re: Small/Mini 10Gbe Router Recommendation

[Hrvoje Popovski]

On 8.4.2021. 22:16, Daniel Melameth wrote:
> On Thu, Apr 8, 2021 at 1:52 PM Hrvoje Popovski  wrote:
>> On 8.4.2021. 20:56, Daniel Melameth wrote:
>>> On Thu, Apr 8, 2021 at 3:57 AM Stuart Henderson  
>>> wrote:
>>>> On 2021-04-07, Daniel Melameth  wrote:
>>>>> Looking to finally part with my legacy OpenBSD router and upgrade to
>>>>> something that can push more than 2Gbps out of a single port.  Since
>>>>> my switching equipment is still only 1Gbe, I also want something that
>>>>> has, at least, two Gbe ports.
>>>>>
>>>>> Any recommendations that work well with OpenBSD?  I am currently
>>>>> thinking 
>>>>> https://www.supermicro.com/en/products/system/Mini-ITX/SYS-E300-8D.cfm,
>>>>> but would like other opinions.
>>
>> my thinking is that if you want to push 10G traffic you'll need at least
>> 8 faster cores ..
>> for now you won't be using them, but when multiqueue RSS forwarding is
>> unlocked you will be happy ...
>>
>> this is vmstat -iz from 12 core box with ixl, mcx and ix
> 
> The dmesg you noted below is for a box with 4 cores, and I was hoping
> to future proof a bit with that.  

dmesg below is from SYS-5018D-FN8T which is basically same box as
SYS-E300-8D only rackmount ..

If I understand you correctly, you
> are saying I'll need 12 cores to do 10Gbps eventually?  What bandwidth
> are you getting out of the box with the dmesg below?

no no, i'm not saying that ... i'm saying that if you want some 10G
router/firewall in the future, you will need more than 4 core, actually
i would suggest 8 or more faster cores with 1 NUMA domain

this vmstat -iz output shows that on 12 cores box only ix is using all
12 queues while mcx and ixl are using 8 queues ... there is explanation
of why 8 queues but i can't remember it .. power of 2 something
something multiqueue :)

regarding forwarding performance of SYS-5018D-FN8T, i can't test it
right now but i remember it was something around 800 or 900 kpps of
plain forwaring and half of that when pf is enabled ..


>> irq114/ixl0270
>> irq115/ixl0:0   40
>> irq116/ixl0:1   00
>> irq117/ixl0:2   00
>> irq118/ixl0:3   00
>> irq119/ixl0:4   00
>> irq120/ixl0:5   00
>> irq121/ixl0:6   00
>> irq122/ixl0:7   80
>> irq123/ixl1270
>> irq124/ixl1:0   40
>> irq125/ixl1:1   00
>> irq126/ixl1:2   00
>> irq127/ixl1:3   00
>> irq128/ixl1:4   00
>> irq129/ixl1:5   00
>> irq130/ixl1:6   00
>> irq131/ixl1:7   80
>> irq132/mcx0350
>> irq133/mcx0:0  110
>> irq134/mcx0:1   00
>> irq135/mcx0:2   00
>> irq136/mcx0:3   00
>> irq137/mcx0:4   00
>> irq138/mcx0:5   00
>> irq139/mcx0:6   00
>> irq140/mcx0:7   00
>> irq141/mcx1390
>> irq142/mcx1:0  110
>> irq143/mcx1:1   00
>> irq144/mcx1:2   00
>> irq145/mcx1:3   00
>> irq146/mcx1:4   00
>> irq147/mcx1:5   00
>> irq148/mcx1:6   00
>> irq149/mcx1:7   00
>> irq150/ix0:0   130
>> irq151/ix0:100
>> irq152/ix0:200
>> irq153/ix0:300
>> irq154/ix0:420
>> irq155/ix0:500
>> irq156/ix0:620
>> irq157/ix0:700
>> irq158/ix0:800
>> irq159/ix0:900
>> irq160/ix0:10   00
>> irq161/ix0:11   00
>> irq162/ix0  

Re: Small/Mini 10Gbe Router Recommendation

[Hrvoje Popovski]

On 8.4.2021. 20:56, Daniel Melameth wrote:
> On Thu, Apr 8, 2021 at 3:57 AM Stuart Henderson  wrote:
>> On 2021-04-07, Daniel Melameth  wrote:
>>> Looking to finally part with my legacy OpenBSD router and upgrade to
>>> something that can push more than 2Gbps out of a single port.  Since
>>> my switching equipment is still only 1Gbe, I also want something that
>>> has, at least, two Gbe ports.
>>>
>>> Any recommendations that work well with OpenBSD?  I am currently
>>> thinking 
>>> https://www.supermicro.com/en/products/system/Mini-ITX/SYS-E300-8D.cfm,
>>> but would like other opinions.
>>
>> I have several routers using that same motherboard (been using them for
>> 3-4 years), they work nicely and have a useful selection of NICs. dmesg 
>> below -
>> the onboard SFP+ are ix0/1, the ixl(4) in there are a PCIE card. DOM works ok
>> on the fibre ports ("ifconfig ix0 sff" etc).
> 
> Wonderful--and the dmesg is even better.
> 

my thinking is that if you want to push 10G traffic you'll need at least
8 faster cores ..
for now you won't be using them, but when multiqueue RSS forwarding is
unlocked you will be happy ...

this is vmstat -iz from 12 core box with ixl, mcx and ix

irq114/ixl0270
irq115/ixl0:0   40
irq116/ixl0:1   00
irq117/ixl0:2   00
irq118/ixl0:3   00
irq119/ixl0:4   00
irq120/ixl0:5   00
irq121/ixl0:6   00
irq122/ixl0:7   80
irq123/ixl1270
irq124/ixl1:0   40
irq125/ixl1:1   00
irq126/ixl1:2   00
irq127/ixl1:3   00
irq128/ixl1:4   00
irq129/ixl1:5   00
irq130/ixl1:6   00
irq131/ixl1:7   80
irq132/mcx0350
irq133/mcx0:0  110
irq134/mcx0:1   00
irq135/mcx0:2   00
irq136/mcx0:3   00
irq137/mcx0:4   00
irq138/mcx0:5   00
irq139/mcx0:6   00
irq140/mcx0:7   00
irq141/mcx1390
irq142/mcx1:0  110
irq143/mcx1:1   00
irq144/mcx1:2   00
irq145/mcx1:3   00
irq146/mcx1:4   00
irq147/mcx1:5   00
irq148/mcx1:6   00
irq149/mcx1:7   00
irq150/ix0:0   130
irq151/ix0:100
irq152/ix0:200
irq153/ix0:300
irq154/ix0:420
irq155/ix0:500
irq156/ix0:620
irq157/ix0:700
irq158/ix0:800
irq159/ix0:900
irq160/ix0:10   00
irq161/ix0:11   00
irq162/ix0  00
irq163/ix1:0   130
irq164/ix1:100
irq165/ix1:220
irq166/ix1:300
irq167/ix1:420
irq168/ix1:500
irq169/ix1:600
irq170/ix1:700
irq171/ix1:800
irq172/ix1:900
irq173/ix1:10   00
irq174/ix1:11   00
irq175/ix1  00




dmesg for this one:
https://www.supermicro.com/en/products/system/1U/5018/SYS-5018D-FN8T.cfm


OpenBSD 6.8-current (GENERIC.MP) #120: Sun Oct 18 09:31:14 MDT 2020
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 17054588928 (16264MB)
avail mem = 16522625024 (15757MB)
random: good seed from bootblocks
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.8 @ 0xed9b0 (47 entries)
bios0: vendor American Megatrends Inc. version "2.1" date 11/08/2019
bios0: Supermicro Super Server
acpi0 at bios0: ACPI 5.0
acpi0: sleep states S0 S4 S5
acpi0: tables DSDT FACP APIC FPDT FIDT SPMI MCFG UEFI DBG2 HPET WDDT
SSDT SSDT SSDT PRAD DMAR HEST BERT ERST EINJ
acpi0: wakeup devices IP2P(S4) EHC1(S4) EHC2(S4) RP07(S4) RP08(S4)
BR1A(S4) BR1B(S4) BR2A(S4) BR2B(S4) BR2C(S4) BR2D(S4) BR3A(S4) BR3B(S4)
BR3C(S4) BR3D(S4) RP01(S

Re: OT: Dell EMC switches

[Hrvoje Popovski]

On 8.4.2021. 20:58, Ivo Chutkin wrote:
> Hello everyone,
> 
> Does anyone have experience with Dell EMS switches?
> 
> Namely S4100 series, S4128F-ON or S4188F-ON.
> 
> Are they robust and reliable?
> 
> I need to replace number of Extreme Networks X650. 10G ports are loaded
> nearly 80% all the time. We are pushing Internet traffic and some
> multicast.
> 
> Also, Dell EMC support third party OS like FTOS, Cumulus Linux OS or Big
> Switch Networks Switch Light. It it means any good.
> 
> Thanks,
> Ivo
> 

Hi,

are you sure that you can put ftos on s4100 series ?
i think that you can put OS10 or something else but not ftos.

i like their VLT (mlag) setup, it's easy to configure and maintain
unlike extreme mlag setup ..

for me OS10 i just not ok :) .. it's debian with lot's of python scripts
but os9 i really nice and mature

if you want i can send you some details privately

Re: pf firewall bridge0 vether0 blocks DHCP for bridge interfaces connected to Windows

[Hrvoje Popovski]

On 10.3.2021. 20:40, da...@hajes.org wrote:
> Hi,
> 
> I did set up OpenBSD router/firewall on PC Engines APU4d4 box.
> 
> First interface is WAN that connects to Internet.
> 
> Remaining three interfaces are bridged with bridge0 via vether0.
> 
> firewall doesn't block LAN/bridge traffic on vether0.
> 
> DHCPD runs on bridge.
> 
> Two Linux hosts (connected to em2 and em3) connect without problem but
> Windows host DHCP requests are blocked on em1.
> 
> I didn't find any info regarding pf and bridging.
> 
> set skip on lo0
> set skip on bridge0
> 
> So far I have found a kludge for Windows "set skip on em1"
> 
> Once, above by line is present in pf.conf, Win 10 host is allowed to
> acquire IP address. Interesting is that Linux has no issues to acquire
> IP addresses via DHCP.
> 
> Any suggestions, please?
> 
> Is it something screwed up in Windows such as short 3-way-handshake?
> 
> 
> Regards
> 
> Hajes
> 

maybe to try veb(4) instead bridge(4) ... in that case use vport instead
vether ..

Re: 10Gbit network work only 1Gbit

[Hrvoje Popovski]

On 26.2.2021. 9:00, csszep wrote:
> Hi!
> 
> I miss something , or veb(4) ifconfig bits not yet commited ?
> 
> OpenBSD 6.9-beta (GENERIC.MP) #358: Wed Feb 24 17:11:53 MST 2021
> dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
> 
> 
>  ifconfig veb0 create
> ifconfig: SIOCIFCREATE: Invalid argument
> 
>


it this latest snapshot ?

Re: 10Gbit network work only 1Gbit

[Hrvoje Popovski]

On 12.11.2019. 10:54, Szél Gábor wrote:
> Dear Hrvoje, Theo,
> 
> Thank you for your answers!
> 
> answers to the questions:
> -  who is parent interface for carp?  -> vlan  ( carp10 interface parent
> vlan10 -> vlan10 interface  parent -> trunk0 )
> - why vlan interfaces don't have ip address ? -> it wasn't needed! i
> think vlan interface need only tag packages. Carp (over vlan) interface
> have IP address.
> - vether implies that you have bridge? -> yes whe have only one bridge
> for bridget openvpn clients, but  we will eliminate it.
> 
> 
> we will do the following:
> - refresh our backup firewall to oBSD 6.6
> - replace trunk interface with aggr
> - remove bridge interface
> 
> if there was an update finised, I'll write again!
> 

Hi,

if you still have bridge and you don't need spanning-tree, try veb
instead. I'm getting 1.95Mpps over veb vs 500Kpps over bridge on 6 x
E5-2643 v2 @ 3.50GHz, 3600.48 MHz.

And of course .. big thanks to dlg@ who wrote it ..

Re: Switching from trunk(4) to aggr(4)

[Hrvoje Popovski]

On 13.12.2020. 23:40, Daniel Jakots wrote:
> I just tried
> # ifconfig aggr0 debug
> # dmesg
> 
> # ifconfig aggr0 down
> # ifconfig aggr0 up
> # ifconfig aggr0 # checked the debug flag was still there
> # dmesg
> 
> 
> I also looked at /var/log/message to be save, but nothing relevant.

Hi,

maybe to put debug in hostname.aggr0 then destroy it and then sh
netstart aggr0 ?

Re: Intl I350 Network Card Not Found

[Hrvoje Popovski]

On 17.9.2020. 20:39, Brandon Woodford wrote:
> Hello,
> 
> I've been trying  to fix an issue with my Intel I350-T4 PCI Network card not 
> being reported to the OpenBSD 6.7 system during boot. Looking through dmesg, 
> I was not able to find any reference to the card or the em interface name 
> that it should have. I've also tried updating all firmware with fw_update. 
> After that I tried creating a /etc/hostname.em1 file that just has dhcp 
> included in it and ran sh /etc/netstart. Unfortunately, no luck as of yet. I 
> was able to find the boot_config(8) man page that describes a similar issue 
> with the ne(4) driver. I went into the boot configuration and ran: find em 
> and received a response of: em* at pci* dev -1 function -1 flags 0x0. Not 
> sure if that means anything.
> 
> Quick note: the card does work on a separate system that is not OpenBSD but 
> FreeBSD.
> 
> Any help in the right direction is appreciated!
> 
> Thanks.
> 

Hi,

i have bunch of i350 cards and they all works perfectly. Can you send dmesg?

Re: openconnect

[Hrvoje Popovski]

On 1.9.2020. 15:22, Stuart Henderson wrote:
> On 2020-09-01, Hrvoje Popovski  wrote:
>> Hi all,
>>
>> does anyone use an openconnect server on openbsd and have guidelines on
>> how to configure it? i see that an openconnect server can use radius, so
>> it's interesting to me. Which client do you use to connect to the
>> openconnect server?
> 
> It worked when I tested after porting ocserv/openconnect, but I'm not using
> it in production. You should be able to connect to ocserv using either the
> openconnect client or cisco anyconnect client.
> 
>> If there is something else that can use radius, i would like to know?
> 
> at least these:
> 
> - npppd (yeuch l2tp :)
> 
> - openvpn (there's a username/pw auth method using a helper script,
> you can write something calling a radius client to check auth, also
> yeuch openvpn :)
> 
> I did once see some code including radius support for iked but it
> was tied up with a bunch of other changes and looked a bit complex
> to separate. I don't recall whether it was just username/pw or if
> it did full EAP.
> 
> 

Tnx for information. It would be great to have radius support for iked
so students could use eduroam username/pass for vpn ...

openconnect

[Hrvoje Popovski]

Hi all,

does anyone use an openconnect server on openbsd and have guidelines on
how to configure it? i see that an openconnect server can use radius, so
it's interesting to me. Which client do you use to connect to the
openconnect server?

If there is something else that can use radius, i would like to know?

Tnx

Re: aggr(4) not working with Intel XXV710 SFP28 on a Supermicro X11DPi-N(T)

[Hrvoje Popovski]

On 17.8.2020. 11:46, Stuart Henderson wrote:
> On 2020-08-15, Hrvoje Popovski  wrote:
>> On 15.8.2020. 0:48, Hrvoje Popovski wrote:
>>> On 12.8.2020. 15:18, Winfred Harrelson wrote:
>>>> On Tue, Aug 11, 2020 at 07:52:10PM +0100, Tom Smyth wrote:
>>>>> Hi Winfred,
>>>>> the intel 710 is a complex card,  I would suggest that you try updating 
>>>>> the
>>>>> firmware on the card, available from intel.com or your card vendor,
>>>>> you may have to boot to a live linux cd to apply the firmware update,
>>>>>
>>>>> but I had some issues with the Intel XL710 cards and I had to update the
>>>>> firmware to get it working stable,
>>>>>
>>>>> I hope this helps
>>>>> Tom Smyth
>>>>
>>>> Adding misc@openbsd.org back to the CC for the record.
>>>>
>>>> Thanks for the quick reply.  I didn't reply back yesterday because I
>>>> was having trouble getting the firmware updated from a Linux boot disk.
>>>> I ended up having to try from a Windows boot disk.  Unfortunately, I
>>>> am getting the same thing again:
>>>>
>>>>
>>>> wharrels@styx2:/home/wharrels# dmesg | grep ^ixl
>>>> ixl0 at pci5 dev 0 function 0 "Intel XXV710 SFP28" rev 0x02: port 0, FW 
>>>> 8.0.61820 API 1.11, msix, 8 queues, address 3c:fd:fe:ed:b7:28
>>>> ixl1 at pci5 dev 0 function 1 "Intel XXV710 SFP28" rev 0x02: port 1, FW 
>>>> 8.0.61820 API 1.11, msix, 8 queues, address 3c:fd:fe:ed:b7:29
>>>> ixl2 at pci8 dev 0 function 0 "Intel XXV710 SFP28" rev 0x02: port 0, FW 
>>>> 8.0.61820 API 1.11, msix, 8 queues, address 3c:fd:fe:eb:19:b0
>>>> ixl3 at pci8 dev 0 function 1 "Intel XXV710 SFP28" rev 0x02: port 1, FW 
>>>> 8.0.61820 API 1.11, msix, 8 queues, address 3c:fd:fe:eb:19:b1
>>>> ixl4 at pci12 dev 0 function 0 "Intel X722 10GBASE-T" rev 0x09: port 0, FW 
>>>> 3.1.57069 API 1.5, msix, 8 queues, address 3c:ec:ef:1a:df:f2
>>>> ixl5 at pci12 dev 0 function 1 "Intel X722 10GBASE-T" rev 0x09: port 1, FW 
>>>> 3.1.57069 API 1.5, msix, 8 queues, address 3c:ec:ef:1a:df:f3
>>>>
>>>> Yup, all the XXV710 cards have been updated to newest firmware.
>>>>
>>>> Now for the (failed) attempt:
>>>>
>>>> wharrels@styx2:/etc# ifconfig ixl0
>>>> ixl0: flags=8843 mtu 1500
>>>> lladdr 3c:fd:fe:ed:b7:28
>>>> index 1 priority 0 llprio 3
>>>> media: Ethernet autoselect (25GbaseSR full-duplex)
>>>> status: active
>>>> wharrels@styx2:/etc# ifconfig ixl2 
>>>> ixl2: flags=8843 mtu 1500
>>>> lladdr 3c:fd:fe:eb:19:b0
>>>> index 3 priority 0 llprio 3
>>>> media: Ethernet autoselect (25GbaseSR full-duplex)
>>>> status: active
>>>> wharrels@styx2:/etc# ifconfig aggr1 create
>>>> wharrels@styx2:/etc# ifconfig aggr1 trunkport ixl0
>>>> wharrels@styx2:/etc# ifconfig aggr1 trunkport ixl2
>>>> wharrels@styx2:/etc# ifconfig aggr1 up
>>>> wharrels@styx2:/etc# ifconfig aggr1
>>>> aggr1: flags=8843 mtu 1500
>>>> lladdr fe:e1:ba:d0:7c:e9
>>>> index 11 priority 0 llprio 7
>>>> trunk: trunkproto lacp
>>>> trunk id: [(8000,fe:e1:ba:d0:7c:e9,000B,,),
>>>>  (,00:00:00:00:00:00,,,)]
>>>> ixl0 lacp actor system pri 0x8000 mac fe:e1:ba:d0:7c:e9, 
>>>> key 0xb, port pri 0x8000 number 0x1
>>>> ixl0 lacp actor state activity,aggregation,defaulted
>>>> ixl0 lacp partner system pri 0x0 mac 00:00:00:00:00:00, 
>>>> key 0x0, port pri 0x0 number 0x0
>>>> ixl0 lacp partner state activity,aggregation,sync
>>>> ixl0 port 
>>>> ixl2 lacp actor system pri 0x8000 mac fe:e1:ba:d0:7c:e9, 
>>>> key 0xb, port pri 0x8000 number 0x3
>>>> ixl2 lacp actor state activity,aggregation,defaulted
>>>> ixl2 lacp partner system pri 0x0 mac 00:00:00:00:00:00, 
>>>> key 0x0, port pri 0x0 number 0x0
>>>> ixl2 lacp partner state activity,aggregation,sync
>>>> ixl2 port 
>>>> groups: aggr
>>>> media: Eth

Re: aggr(4) not working with Intel XXV710 SFP28 on a Supermicro X11DPi-N(T)

[Hrvoje Popovski]

On 15.8.2020. 0:48, Hrvoje Popovski wrote:
> On 12.8.2020. 15:18, Winfred Harrelson wrote:
>> On Tue, Aug 11, 2020 at 07:52:10PM +0100, Tom Smyth wrote:
>>> Hi Winfred,
>>> the intel 710 is a complex card,  I would suggest that you try updating the
>>> firmware on the card, available from intel.com or your card vendor,
>>> you may have to boot to a live linux cd to apply the firmware update,
>>>
>>> but I had some issues with the Intel XL710 cards and I had to update the
>>> firmware to get it working stable,
>>>
>>> I hope this helps
>>> Tom Smyth
>>
>> Adding misc@openbsd.org back to the CC for the record.
>>
>> Thanks for the quick reply.  I didn't reply back yesterday because I
>> was having trouble getting the firmware updated from a Linux boot disk.
>> I ended up having to try from a Windows boot disk.  Unfortunately, I
>> am getting the same thing again:
>>
>>
>> wharrels@styx2:/home/wharrels# dmesg | grep ^ixl
>> ixl0 at pci5 dev 0 function 0 "Intel XXV710 SFP28" rev 0x02: port 0, FW 
>> 8.0.61820 API 1.11, msix, 8 queues, address 3c:fd:fe:ed:b7:28
>> ixl1 at pci5 dev 0 function 1 "Intel XXV710 SFP28" rev 0x02: port 1, FW 
>> 8.0.61820 API 1.11, msix, 8 queues, address 3c:fd:fe:ed:b7:29
>> ixl2 at pci8 dev 0 function 0 "Intel XXV710 SFP28" rev 0x02: port 0, FW 
>> 8.0.61820 API 1.11, msix, 8 queues, address 3c:fd:fe:eb:19:b0
>> ixl3 at pci8 dev 0 function 1 "Intel XXV710 SFP28" rev 0x02: port 1, FW 
>> 8.0.61820 API 1.11, msix, 8 queues, address 3c:fd:fe:eb:19:b1
>> ixl4 at pci12 dev 0 function 0 "Intel X722 10GBASE-T" rev 0x09: port 0, FW 
>> 3.1.57069 API 1.5, msix, 8 queues, address 3c:ec:ef:1a:df:f2
>> ixl5 at pci12 dev 0 function 1 "Intel X722 10GBASE-T" rev 0x09: port 1, FW 
>> 3.1.57069 API 1.5, msix, 8 queues, address 3c:ec:ef:1a:df:f3
>>
>> Yup, all the XXV710 cards have been updated to newest firmware.
>>
>> Now for the (failed) attempt:
>>
>> wharrels@styx2:/etc# ifconfig ixl0
>> ixl0: flags=8843 mtu 1500
>> lladdr 3c:fd:fe:ed:b7:28
>> index 1 priority 0 llprio 3
>> media: Ethernet autoselect (25GbaseSR full-duplex)
>> status: active
>> wharrels@styx2:/etc# ifconfig ixl2 
>> ixl2: flags=8843 mtu 1500
>> lladdr 3c:fd:fe:eb:19:b0
>> index 3 priority 0 llprio 3
>> media: Ethernet autoselect (25GbaseSR full-duplex)
>> status: active
>> wharrels@styx2:/etc# ifconfig aggr1 create
>> wharrels@styx2:/etc# ifconfig aggr1 trunkport ixl0
>> wharrels@styx2:/etc# ifconfig aggr1 trunkport ixl2
>> wharrels@styx2:/etc# ifconfig aggr1 up
>> wharrels@styx2:/etc# ifconfig aggr1
>> aggr1: flags=8843 mtu 1500
>> lladdr fe:e1:ba:d0:7c:e9
>> index 11 priority 0 llprio 7
>> trunk: trunkproto lacp
>> trunk id: [(8000,fe:e1:ba:d0:7c:e9,000B,,),
>>  (,00:00:00:00:00:00,,,)]
>> ixl0 lacp actor system pri 0x8000 mac fe:e1:ba:d0:7c:e9, key 
>> 0xb, port pri 0x8000 number 0x1
>> ixl0 lacp actor state activity,aggregation,defaulted
>> ixl0 lacp partner system pri 0x0 mac 00:00:00:00:00:00, key 
>> 0x0, port pri 0x0 number 0x0
>> ixl0 lacp partner state activity,aggregation,sync
>> ixl0 port 
>> ixl2 lacp actor system pri 0x8000 mac fe:e1:ba:d0:7c:e9, key 
>> 0xb, port pri 0x8000 number 0x3
>> ixl2 lacp actor state activity,aggregation,defaulted
>> ixl2 lacp partner system pri 0x0 mac 00:00:00:00:00:00, key 
>> 0x0, port pri 0x0 number 0x0
>> ixl2 lacp partner state activity,aggregation,sync
>> ixl2 port 
>> groups: aggr
>> media: Ethernet autoselect
>> status: no carrier
>>
>>
>>
>> I tried doing another sysupgrade this morning just in case something
>> had changed overnight but no luck.  Any other ideas?
>>
>> Winfred
>>
> 
> Hi,
> 
> could you try install snapshot from http://ftp.hostserver.de/archive/
> that is older than Thu Jun 25 06:41:38 2020 UTC ...
> 
> maybe this commit broke xxv710
> http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/sys/dev/pci/if_ixl.c?rev=1.56&content-type=text/x-cvsweb-markup
> 
> i have vlans over aggr over x710-da2 with latest snapshot and it's
> working as expected ..
> 
> ixl0 at pci1 dev 0 function 0 "Intel X710 SFP+" rev 0x02: port 0, FW
> 7.3.60988 API 1.10, msix, 8 queues
> ixl1 at pci1 dev 0 function 1 "Intel X710 SFP+" rev 0x02: port 1, FW
> 7.3.60988 API 1.10, msix, 8 queues
> 

with new firmware aggr is working

ixl0 at pci1 dev 0 function 0 "Intel X710 SFP+" rev 0x02: port 0, FW
8.0.61820 API 1.11, msix, 8 queues
ixl1 at pci1 dev 0 function 1 "Intel X710 SFP+" rev 0x02: port 1, FW
8.0.61820 API 1.11, msix, 8 queues

Re: aggr(4) not working with Intel XXV710 SFP28 on a Supermicro X11DPi-N(T)

[Hrvoje Popovski]

On 12.8.2020. 15:18, Winfred Harrelson wrote:
> On Tue, Aug 11, 2020 at 07:52:10PM +0100, Tom Smyth wrote:
>> Hi Winfred,
>> the intel 710 is a complex card,  I would suggest that you try updating the
>> firmware on the card, available from intel.com or your card vendor,
>> you may have to boot to a live linux cd to apply the firmware update,
>>
>> but I had some issues with the Intel XL710 cards and I had to update the
>> firmware to get it working stable,
>>
>> I hope this helps
>> Tom Smyth
> 
> Adding misc@openbsd.org back to the CC for the record.
> 
> Thanks for the quick reply.  I didn't reply back yesterday because I
> was having trouble getting the firmware updated from a Linux boot disk.
> I ended up having to try from a Windows boot disk.  Unfortunately, I
> am getting the same thing again:
> 
> 
> wharrels@styx2:/home/wharrels# dmesg | grep ^ixl
> ixl0 at pci5 dev 0 function 0 "Intel XXV710 SFP28" rev 0x02: port 0, FW 
> 8.0.61820 API 1.11, msix, 8 queues, address 3c:fd:fe:ed:b7:28
> ixl1 at pci5 dev 0 function 1 "Intel XXV710 SFP28" rev 0x02: port 1, FW 
> 8.0.61820 API 1.11, msix, 8 queues, address 3c:fd:fe:ed:b7:29
> ixl2 at pci8 dev 0 function 0 "Intel XXV710 SFP28" rev 0x02: port 0, FW 
> 8.0.61820 API 1.11, msix, 8 queues, address 3c:fd:fe:eb:19:b0
> ixl3 at pci8 dev 0 function 1 "Intel XXV710 SFP28" rev 0x02: port 1, FW 
> 8.0.61820 API 1.11, msix, 8 queues, address 3c:fd:fe:eb:19:b1
> ixl4 at pci12 dev 0 function 0 "Intel X722 10GBASE-T" rev 0x09: port 0, FW 
> 3.1.57069 API 1.5, msix, 8 queues, address 3c:ec:ef:1a:df:f2
> ixl5 at pci12 dev 0 function 1 "Intel X722 10GBASE-T" rev 0x09: port 1, FW 
> 3.1.57069 API 1.5, msix, 8 queues, address 3c:ec:ef:1a:df:f3
> 
> Yup, all the XXV710 cards have been updated to newest firmware.
> 
> Now for the (failed) attempt:
> 
> wharrels@styx2:/etc# ifconfig ixl0
> ixl0: flags=8843 mtu 1500
> lladdr 3c:fd:fe:ed:b7:28
> index 1 priority 0 llprio 3
> media: Ethernet autoselect (25GbaseSR full-duplex)
> status: active
> wharrels@styx2:/etc# ifconfig ixl2 
> ixl2: flags=8843 mtu 1500
> lladdr 3c:fd:fe:eb:19:b0
> index 3 priority 0 llprio 3
> media: Ethernet autoselect (25GbaseSR full-duplex)
> status: active
> wharrels@styx2:/etc# ifconfig aggr1 create
> wharrels@styx2:/etc# ifconfig aggr1 trunkport ixl0
> wharrels@styx2:/etc# ifconfig aggr1 trunkport ixl2
> wharrels@styx2:/etc# ifconfig aggr1 up
> wharrels@styx2:/etc# ifconfig aggr1
> aggr1: flags=8843 mtu 1500
> lladdr fe:e1:ba:d0:7c:e9
> index 11 priority 0 llprio 7
> trunk: trunkproto lacp
> trunk id: [(8000,fe:e1:ba:d0:7c:e9,000B,,),
>  (,00:00:00:00:00:00,,,)]
> ixl0 lacp actor system pri 0x8000 mac fe:e1:ba:d0:7c:e9, key 
> 0xb, port pri 0x8000 number 0x1
> ixl0 lacp actor state activity,aggregation,defaulted
> ixl0 lacp partner system pri 0x0 mac 00:00:00:00:00:00, key 
> 0x0, port pri 0x0 number 0x0
> ixl0 lacp partner state activity,aggregation,sync
> ixl0 port 
> ixl2 lacp actor system pri 0x8000 mac fe:e1:ba:d0:7c:e9, key 
> 0xb, port pri 0x8000 number 0x3
> ixl2 lacp actor state activity,aggregation,defaulted
> ixl2 lacp partner system pri 0x0 mac 00:00:00:00:00:00, key 
> 0x0, port pri 0x0 number 0x0
> ixl2 lacp partner state activity,aggregation,sync
> ixl2 port 
> groups: aggr
> media: Ethernet autoselect
> status: no carrier
> 
> 
> 
> I tried doing another sysupgrade this morning just in case something
> had changed overnight but no luck.  Any other ideas?
> 
> Winfred
> 

Hi,

could you try install snapshot from http://ftp.hostserver.de/archive/
that is older than Thu Jun 25 06:41:38 2020 UTC ...

maybe this commit broke xxv710
http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/sys/dev/pci/if_ixl.c?rev=1.56&content-type=text/x-cvsweb-markup

i have vlans over aggr over x710-da2 with latest snapshot and it's
working as expected ..

ixl0 at pci1 dev 0 function 0 "Intel X710 SFP+" rev 0x02: port 0, FW
7.3.60988 API 1.10, msix, 8 queues
ixl1 at pci1 dev 0 function 1 "Intel X710 SFP+" rev 0x02: port 1, FW
7.3.60988 API 1.10, msix, 8 queues

could  you send output from these two commands ..
ifconfig ixl sff
ifconfig ixl media

Re: IPSec heavy traffic slows down all network traffic

[Hrvoje Popovski]

On 17.7.2020. 20:17, jean-yves boisiaud wrote:
> hello,
> 
> Last week, I upgraded a couple of firewalls using carp/pfsync and sasyncd
> from 6.0 to 6.7 (yes, big jump !).
> 
> I also applied all the 6.7 published patches.
> 
> When some heavy traffic takes one of the IPSec tunnel, I noticed that :
> - all network connections are slowed down
> - unused network bandwidth increase instead of decrease
> - idle CPU move towards 0, and spinning increase to take about 50% of the
> CPU
> 
> When I stop the IPSec traffic :
> - network connections increase immediatly
> - unused network bandwidth cecreases immediately
> - spinning CPU is low.
> 
> Yes I know, my hardware is a bit old. I understand that CPU raises due to
> IPSec crypto, but I do not understand why network performance decrease.


maybe intel mitigation stuff decreased your performance. it in from
openbsd 6.3 ...
don't know if you are using aes for ipsec, but you cpu doesn't have
aes-ni... maybe to try wireguard ? :)

Re: supermicro - A2SDV-8C-LN8F

[Hrvoje Popovski]

On 11.7.2020. 11:13, mlopenb...@xiphosura.co.uk wrote:
> On Sat, 11 Jul 2020 00:13:34 +0200
> Hrvoje Popovski  wrote:
> 
>> Hi all,
>>
>> does anyone have experience or dmesg of this motherboard
>> https://www.supermicro.com/en/products/motherboard/A2SDV-8C-LN8F
>>
>> is it stable? i'm most interested in network performance and network
>> cards. in motherboard manual i couldn't find what "Quad LAN with
>> Intel® C3000 SoC" means ?  is it i350 em(4)?
>>
>>
>> Thank you ..
>>
> 
> Hello Hrvoje,
> 
> I am using the smaller but similar Supermicro A2SDi-4C-HLN4F which also
> uses thethe Intel C3000 SoC.  (In the manual it is described as
> "Intel® Atom SoC C3000 Series (FCBGA1310) Processor").
> 
> I have been using two systems since 6.7 was released (the first to
> support the onboard NICs) and they have been perfectly stable.
> 
> I did some testing with a 6.7 pre-release (in March) and using
> tcpbench(1) and a direct cable connection I was getting about 940 Mbps
> between two systems.
> 
> The onboard NICs are detected as ix(4) "Intel X553 SGMII"
>

Thank you guys ... i totally forgot that it's ix although it's 1Gbps

supermicro - A2SDV-8C-LN8F

[Hrvoje Popovski]

Hi all,

does anyone have experience or dmesg of this motherboard
https://www.supermicro.com/en/products/motherboard/A2SDV-8C-LN8F

is it stable? i'm most interested in network performance and network
cards. in motherboard manual i couldn't find what "Quad LAN with Intel®
C3000 SoC" means ?  is it i350 em(4)?


Thank you ..

Re: strongSwan cannot install IPsec policies on OpenBSD

[Hrvoje Popovski]

On 20.2.2020. 18:47, Peter Müller wrote:
> Hello openbsd-misc,
> 
> is anybody out there running strongSwan as an IPsec client for a net-to-net 
> connection
> on an OpenBSD machine?
> 
> If so, I would be very grateful to know which steps are necessary in order to 
> successfully
> route traffic through this n2n connection and what your ipsec.conf file (and 
> other ones,
> if necessary) looks like.
> 
> Sorry for bringing this up again, but I am out of ideas now and packaging 
> strongSwan
> for OpenBSD would not make sense if it could not be used properly. :-)
> 
> Thanks again for any advice on this.
> 
> Best regards,
> Peter Müller
> 

Maybe stupid question... can you use isakmpd on openbsd box and
strongswan on that other box ? i have working configuration for
site-to-site setup and it's working quite well ..

Re: Brand new server - bad adventures

[Hrvoje Popovski]

On 22.1.2020. 21:30, Özgür Kazancci wrote:
> Hello everyone! Greetings to misc people!
> 
> Got a brand new dedicated server with a hardware: Intel Xeon-E 2274G -
> 64GB DDR4 ECC 2666MHz - 2x SSD NVMe 960GB
> and installed "brand new" OpenBSD 6.6 on it. (I'm managing it remotely
> via KVM/IPMI)


Hi,

could you install snapshot on this box and if problem is still there
send report to bugs@openbsd
https://www.openbsd.org/report.html

at least in report send "sendbug -P" from that box

Re: small aggr problem ( on current )

[Hrvoje Popovski]

On 15.12.2019. 23:01, Hrvoje Popovski wrote:
> On 15.12.2019. 12:45, Holger Glaess wrote:
>> hi
>>
>>
>>   runing version
>>
>>
>> /etc 16>dmesg | more
>> Copyright (c) 1982, 1986, 1989, 1991, 1993
>>     The Regents of the University of California.  All rights reserved.
>> Copyright (c) 1995-2019 OpenBSD. All rights reserved.
>> https://www.OpenBSD.org
>>
>> OpenBSD 6.6-current (GENERIC.MP) #48: Tue Dec 10 16:30:01 MST 2019
>> dera...@octeon.openbsd.org:/usr/src/sys/arch/octeon/compile/GENERIC.MP
>>
>>
>>
>> after a reboot the aggr interface do not aggregate the connection with
>> the switch,
>>
>> just after an physical disaconnection from the ethernet cable , wait for
>> some sec,
>>
>> and replugin .
>>
>>
>> the the iterface are up and active, before ifconfig says "no carrier"
>> but the interfaces have
>>
>> carrier.
>>
>> i dont have the problem with the trunk interface on the same hardware.
>>
>>
>> you are on bellab as root
>> /etc 20>cat /etc/hostname.cnmac1
>> mtu 1518
>> up
>>
>> 12:43:59 Sun Dec 15
>> you are on bellab as root
>> /etc 21>cat /etc/hostname.cnmac2
>> mtu 1518
>> up
>>
>> 12:44:01 Sun Dec 15
>> you are on bellab as root
>> /etc 22>cat /etc/hostname.aggr0
>> trunkport cnmac1
>> trunkport cnmac2
>> mtu 1518
>> up
>>
>>
>> holger
>>
>>
>>
> Hi,
> 
> maybe logs below would help for further troubleshooting because i'm
> seeing same behavior.
> 
> when i add debug statement in hostname.agg0 and boot box i'm getting
> this log
> 
> starting network
> aggr0 ix0 rxm: LACP_DISABLED (LACP_Enabled) -> PORT_DISABLED
> aggr0 ix0: selection logic: unselected (rxm !CURRENT)
> aggr0 ix1 rxm: LACP_DISABLED (LACP_Enabled) -> PORT_DISABLED
> aggr0 ix1: selection logic: unselected (rxm !CURRENT)
> aggr0 ix2 rxm: LACP_DISABLED (LACP_Enabled) -> PORT_DISABLED
> aggr0 ix2: selection logic: unselected (rxm !CURRENT)
> aggr0 ix3 rxm: LACP_DISABLED (LACP_Enabled) -> PORT_DISABLED
> aggr0 ix3: selection logic: unselected (rxm !CURRENT)
> reordering libraries: done.
> 
> after boot aggr status is "no carrier"
> sh /etc/netstart isn't helping
> 
> but with ifconfig ix0-ix4 down/up aggr interface start to work normally
> 
> log when doing ifconfig ix0-ix4 down/up


just a little follow up:

i've tested aggr on two boxes. first box is dell r620 and second one is
supermicro SYS-5018D-FN8T. both boxes are connected to dell s4810
switch. Same cables, same ports, same port-channles on switch, timeout
fast or slow, both with ix 82599 interfaces ... (x552 ix interfaces are
disabled on supermicro box) ...

r620 is working without any problems and supermicro box is having same
problem as described above...

trunk interface are working on both boxes without any problem ..


this is fun :)

Re: small aggr problem ( on current )

[Hrvoje Popovski]

On 15.12.2019. 12:45, Holger Glaess wrote:
> hi
> 
> 
>   runing version
> 
> 
> /etc 16>dmesg | more
> Copyright (c) 1982, 1986, 1989, 1991, 1993
>     The Regents of the University of California.  All rights reserved.
> Copyright (c) 1995-2019 OpenBSD. All rights reserved.
> https://www.OpenBSD.org
> 
> OpenBSD 6.6-current (GENERIC.MP) #48: Tue Dec 10 16:30:01 MST 2019
> dera...@octeon.openbsd.org:/usr/src/sys/arch/octeon/compile/GENERIC.MP
> 
> 
> 
> after a reboot the aggr interface do not aggregate the connection with
> the switch,
> 
> just after an physical disaconnection from the ethernet cable , wait for
> some sec,
> 
> and replugin .
> 
> 
> the the iterface are up and active, before ifconfig says "no carrier"
> but the interfaces have
> 
> carrier.
> 
> i dont have the problem with the trunk interface on the same hardware.
> 
> 
> you are on bellab as root
> /etc 20>cat /etc/hostname.cnmac1
> mtu 1518
> up
> 
> 12:43:59 Sun Dec 15
> you are on bellab as root
> /etc 21>cat /etc/hostname.cnmac2
> mtu 1518
> up
> 
> 12:44:01 Sun Dec 15
> you are on bellab as root
> /etc 22>cat /etc/hostname.aggr0
> trunkport cnmac1
> trunkport cnmac2
> mtu 1518
> up
> 
> 
> holger
> 
> 
> 

Hi,

maybe logs below would help for further troubleshooting because i'm
seeing same behavior.

when i add debug statement in hostname.agg0 and boot box i'm getting
this log

starting network
aggr0 ix0 rxm: LACP_DISABLED (LACP_Enabled) -> PORT_DISABLED
aggr0 ix0: selection logic: unselected (rxm !CURRENT)
aggr0 ix1 rxm: LACP_DISABLED (LACP_Enabled) -> PORT_DISABLED
aggr0 ix1: selection logic: unselected (rxm !CURRENT)
aggr0 ix2 rxm: LACP_DISABLED (LACP_Enabled) -> PORT_DISABLED
aggr0 ix2: selection logic: unselected (rxm !CURRENT)
aggr0 ix3 rxm: LACP_DISABLED (LACP_Enabled) -> PORT_DISABLED
aggr0 ix3: selection logic: unselected (rxm !CURRENT)
reordering libraries: done.

after boot aggr status is "no carrier"
sh /etc/netstart isn't helping

but with ifconfig ix0-ix4 down/up aggr interface start to work normally

log when doing ifconfig ix0-ix4 down/up

aggr0 ix0 rxm: PORT_DISABLED (port_enabled) -> EXPIRED
aggr0 ix0 rxm: EXPIRED (LACPDU) -> CURRENT
aggr0 ix0: Selected UNSELECTED -> SELECTED
aggr0 ix0 mux: DETACHED (Selected == SELECTED) -> WAITING
aggr0 ix0 mux: WAITING (Selected == SELECTED) -> ATTACHED
aggr0 ix0: mux attached
aggr0 ix1 rxm: PORT_DISABLED (port_enabled) -> EXPIRED
aggr0 ix0 mux: ATTACHED (Partner.Sync) -> COLLECTING
aggr0 ix0: collecting enabled
aggr0 ix0 mux: COLLECTING (Partner.Sync) -> DISTRIBUTING
aggr0 ix0: distributing enabled
aggr0 ix1 rxm: EXPIRED (LACPDU) -> CURRENT
aggr0 ix1: Selected UNSELECTED -> SELECTED
aggr0 ix1 mux: DETACHED (Selected == SELECTED) -> WAITING
aggr0 ix1 mux: WAITING (Selected == SELECTED) -> ATTACHED
aggr0 ix1: mux attached
aggr0 ix2 rxm: PORT_DISABLED (port_enabled) -> EXPIRED
aggr0 ix2 rxm: EXPIRED (LACPDU) -> CURRENT
aggr0 ix2: Selected UNSELECTED -> SELECTED
aggr0 ix2 mux: DETACHED (Selected == SELECTED) -> WAITING
aggr0 ix2 mux: WAITING (Selected == SELECTED) -> ATTACHED
aggr0 ix2: mux attached
aggr0 ix3 rxm: PORT_DISABLED (port_enabled) -> EXPIRED
aggr0 ix3 rxm: EXPIRED (LACPDU) -> CURRENT
aggr0 ix3: Selected UNSELECTED -> SELECTED
aggr0 ix3 mux: DETACHED (Selected == SELECTED) -> WAITING
aggr0 ix3 mux: WAITING (Selected == SELECTED) -> ATTACHED
aggr0 ix3: mux attached
aggr0 ix1 mux: ATTACHED (Partner.Sync) -> COLLECTING
aggr0 ix1: collecting enabled
aggr0 ix1 mux: COLLECTING (Partner.Sync) -> DISTRIBUTING
aggr0 ix1: distributing enabled
aggr0 ix2 mux: ATTACHED (Partner.Sync) -> COLLECTING
aggr0 ix2: collecting enabled
aggr0 ix2 mux: COLLECTING (Partner.Sync) -> DISTRIBUTING
aggr0 ix2: distributing enabled
aggr0 ix3 mux: ATTACHED (Partner.Sync) -> COLLECTING
aggr0 ix3: collecting enabled
aggr0 ix3 mux: COLLECTING (Partner.Sync) -> DISTRIBUTING
aggr0 ix3: distributing enabled

Re: issues configuring vlan on top of aggr device

[Hrvoje Popovski]

On 3.12.2019. 15:11, Pedro Caetano wrote:
> Hi again,
> 
> I'm sorry, but since the boxes do not (yet) have working networking it
> is not easy for me to get the text output.
> I'm attaching a few pictures with the requested output.
> 
> https://picpaste.me/images/2019/12/03/cat_hostname.vl3800_hostname.aggr0.jpg
> https://picpaste.me/images/2019/12/03/ifconfig_vl3800.jpg

you should have ip address on vlan3800 interface, right?

> https://picpaste.me/images/2019/12/03/ifconfig_aggr0.jpg

Re: issues configuring vlan on top of aggr device

[Hrvoje Popovski]

On 3.12.2019. 13:15, Pedro Caetano wrote:
> Hi Hrvoje, thank you for the fast reply,
> 
> Unfortunately I have the same behavior.
> The aggr0 works as expected, as I can see the links bonded on the switch.
> I'm able to se the correct vid s, when tcpdump'ing the aggr0 interface.
> 
> I'd appreciate any help on this topic.
> 

can you send ifconfig aggr0 and ifconfig vlan3800 ?




> This configuration is working on -current with em(4) nics.
> 
> 
> Best regards,
> Pedro Caetano
> 
> A terça, 3/12/2019, 12:01, Hrvoje Popovski  <mailto:hrv...@srce.hr>> escreveu:
> 
> On 3.12.2019. 12:21, Pedro Caetano wrote:
> > Hi misc@
> >
> > I'm running openbsd 6.6 with latest patches running on a pair of
> hp dl 360
> > gen6 servers.
> >
> > I'm attempting to configure an aggr0 device towards a cat 3650.
> >
> > The aggr0 associates successfully with the switch, but I'm unable
> to run
> > vlans on top of it.
> >
> > The configuration on openbsd is the following:
> > #ifconfig aggr0 create
> > #ifconfig aggr0 trunkport bnx0
> > #ifconfig aggr0 trunkport bnx1
> 
> add this - ifconfig aggr0 up
> if you have hostname.aggr0 add "up" at the end of that file ...
> 
> > #ifconfig vlan3800 create
> > #ifconfig vlan3800 vnetid 3800
> > #ifconfig vlan3800 parent aggr0
> > #ifconfig vlan3800 10.80.253.10/24 <http://10.80.253.10/24>
> > ifconfig: SIOCAIFADDR: No buffer space available.
> 

Re: issues configuring vlan on top of aggr device

[Hrvoje Popovski]

On 3.12.2019. 12:21, Pedro Caetano wrote:
> Hi misc@
> 
> I'm running openbsd 6.6 with latest patches running on a pair of hp dl 360
> gen6 servers.
> 
> I'm attempting to configure an aggr0 device towards a cat 3650.
> 
> The aggr0 associates successfully with the switch, but I'm unable to run
> vlans on top of it.
> 
> The configuration on openbsd is the following:
> #ifconfig aggr0 create
> #ifconfig aggr0 trunkport bnx0
> #ifconfig aggr0 trunkport bnx1

add this - ifconfig aggr0 up
if you have hostname.aggr0 add "up" at the end of that file ...

> #ifconfig vlan3800 create
> #ifconfig vlan3800 vnetid 3800
> #ifconfig vlan3800 parent aggr0
> #ifconfig vlan3800 10.80.253.10/24
> ifconfig: SIOCAIFADDR: No buffer space available.

Re: 10Gbit network work only 1Gbit

[Hrvoje Popovski]

On 13.11.2019. 16:37, Gregory Edigarov wrote:
> could you please do one more test:
> "forwarding over ix0 and ix1, pf enabled, 5 tcp states"

with this generator i can't use tcp. generally pps with 5 or 50
states are more or less same ... problem with tcp testing is that i
can't get precise pps numbers ...

and only for you :)
with iperf3 (8 tcp streams) on client boxes i'm getting this results ...

forwarding over ix0 and ix1, pf and ipsec disabled
9.40Gbps

forwarding over ix0 and ix1, pf enabled, 8 tcp streams
7.40Gbps

forwarding over ix0 and ix1, ipsec established over em0, pf disabled
8.10Gbps

forwarding over ix0 and ix1, ipsec established over em0, pf enabled, 8
TCP streams
5.25Gbps


> On 13.11.19 12:52, Hrvoje Popovski wrote:
>> On 13.11.2019. 10:59, Hrvoje Popovski wrote:
>>> On 12.11.2019. 10:54, Szél Gábor wrote:
>>>> Dear Hrvoje, Theo,
>>>>
>>>> Thank you for your answers!
>>>>
>>>> answers to the questions:
>>>> -  who is parent interface for carp?  -> vlan  ( carp10 interface
>>>> parent
>>>> vlan10 -> vlan10 interface  parent -> trunk0 )
>>>> - why vlan interfaces don't have ip address ? -> it wasn't needed! i
>>>> think vlan interface need only tag packages. Carp (over vlan) interface
>>>> have IP address.
>>> it's little strange to me to not have ip address on parent carp
>>> interface, but if it works for you ... ok..
>>>
>>>> - vether implies that you have bridge? -> yes whe have only one bridge
>>>> for bridget openvpn clients, but  we will eliminate it.
>>>>
>>>>
>>>> we will do the following:
>>>> - refresh our backup firewall to oBSD 6.6
>>>> - replace trunk interface with aggr
>>>> - remove bridge interface
>>> this is nice start to make you setup faster. big performance killer in
>>> your setup is ipsec and old hardware. maybe oce(4) but i never tested
>>> it, so i'm not sure ... if you can, change oce with ix, intel x520 is
>>> not that expensive ..
>>>
>>> bridge is slow, but only for traffic that goes through it. with ipsec,
>>> the same second when tunnel is established, forwarding performance will
>>> drop significantly on whole firewall ...
>>
>> i forgot numbers, so i did quick tests ..
>>
>>
>> forwarding over ix0 and ix1, pf and ipsec disabled
>> 1.35Mpps
>>
>> forwarding over ix0 and ix1, pf enabled, 500 UDP states
>> 800Kpps
>>
>> forwarding over ix0 and ix1, ipsec established over em0, pf disabled
>> 800Kpps
>>
>> forwarding over ix0 and ix1, ipsec established over em0, pf enabled, 500
>> UDP states
>> 550Kpps
>>
>>
>>
>> OpenBSD 6.6-current (GENERIC.MP) #453: Mon Nov 11 21:40:31 MST 2019
>>  dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
>> real mem = 17115840512 (16322MB)
>> avail mem = 16584790016 (15816MB)
>> mpath0 at root
>> scsibus0 at mpath0: 256 targets
>> mainbus0 at root
>> bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xcf42c000 (99 entries)
>> bios0: vendor Dell Inc. version "2.8.0" date 06/26/2019
>> bios0: Dell Inc. PowerEdge R620
>> acpi0 at bios0: ACPI 3.0
>> acpi0: sleep states S0 S4 S5
>> acpi0: tables DSDT FACP APIC SPCR HPET DMAR MCFG WD__ SLIC ERST HEST
>> BERT EINJ TCPA PC__ SRAT SSDT
>> acpi0: wakeup devices PCI0(S5)
>> acpitimer0 at acpi0: 3579545 Hz, 24 bits
>> acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
>> cpu0 at mainbus0: apid 4 (boot processor)
>> cpu0: Intel(R) Xeon(R) CPU E5-2643 v2 @ 3.50GHz, 3600.53 MHz, 06-3e-04
>> cpu0:
>> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
>>
>> cpu0: 256KB 64b/line 8-way L2 cache
>> cpu0: smt 0, core 2, package 0
>> mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
>> cpu0: apic clock running at 100MHz
>> cpu0: mwait min=64, max=64, C-substates=0.2.1.1, IBE
>> cpu1 at mainbus0: apid 6 (application processor)
>> cpu1: Intel(R) Xeon(R) CPU E5-2643 v2 @ 3.50GHz, 3600.01 MHz, 06-3e-04
>> cpu1:
>> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLM

Re: 10Gbit network work only 1Gbit

[Hrvoje Popovski]

On 13.11.2019. 10:59, Hrvoje Popovski wrote:
> On 12.11.2019. 10:54, Szél Gábor wrote:
>> Dear Hrvoje, Theo,
>>
>> Thank you for your answers!
>>
>> answers to the questions:
>> -  who is parent interface for carp?  -> vlan  ( carp10 interface parent
>> vlan10 -> vlan10 interface  parent -> trunk0 )
>> - why vlan interfaces don't have ip address ? -> it wasn't needed! i
>> think vlan interface need only tag packages. Carp (over vlan) interface
>> have IP address.
> 
> it's little strange to me to not have ip address on parent carp
> interface, but if it works for you ... ok..
> 
>> - vether implies that you have bridge? -> yes whe have only one bridge
>> for bridget openvpn clients, but  we will eliminate it.
>>
>>
>> we will do the following:
>> - refresh our backup firewall to oBSD 6.6
>> - replace trunk interface with aggr
>> - remove bridge interface
> 
> this is nice start to make you setup faster. big performance killer in
> your setup is ipsec and old hardware. maybe oce(4) but i never tested
> it, so i'm not sure ... if you can, change oce with ix, intel x520 is
> not that expensive ..
> 
> bridge is slow, but only for traffic that goes through it. with ipsec,
> the same second when tunnel is established, forwarding performance will
> drop significantly on whole firewall ...


i forgot numbers, so i did quick tests ..


forwarding over ix0 and ix1, pf and ipsec disabled
1.35Mpps

forwarding over ix0 and ix1, pf enabled, 500 UDP states
800Kpps

forwarding over ix0 and ix1, ipsec established over em0, pf disabled
800Kpps

forwarding over ix0 and ix1, ipsec established over em0, pf enabled, 500
UDP states
550Kpps



OpenBSD 6.6-current (GENERIC.MP) #453: Mon Nov 11 21:40:31 MST 2019
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 17115840512 (16322MB)
avail mem = 16584790016 (15816MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xcf42c000 (99 entries)
bios0: vendor Dell Inc. version "2.8.0" date 06/26/2019
bios0: Dell Inc. PowerEdge R620
acpi0 at bios0: ACPI 3.0
acpi0: sleep states S0 S4 S5
acpi0: tables DSDT FACP APIC SPCR HPET DMAR MCFG WD__ SLIC ERST HEST
BERT EINJ TCPA PC__ SRAT SSDT
acpi0: wakeup devices PCI0(S5)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 4 (boot processor)
cpu0: Intel(R) Xeon(R) CPU E5-2643 v2 @ 3.50GHz, 3600.53 MHz, 06-3e-04
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 2, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 100MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.1, IBE
cpu1 at mainbus0: apid 6 (application processor)
cpu1: Intel(R) Xeon(R) CPU E5-2643 v2 @ 3.50GHz, 3600.01 MHz, 06-3e-04
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 3, package 0
cpu2 at mainbus0: apid 8 (application processor)
cpu2: Intel(R) Xeon(R) CPU E5-2643 v2 @ 3.50GHz, 3600.01 MHz, 06-3e-04
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 4, package 0
cpu3 at mainbus0: apid 16 (application processor)
cpu3: Intel(R) Xeon(R) CPU E5-2643 v2 @ 3.50GHz, 3600.01 MHz, 06-3e-04
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu3: 256KB 64b/line 8-way L2 cache
cpu3: smt 0, core 8, package 0
cpu4 at ma

Re: 10Gbit network work only 1Gbit

[Hrvoje Popovski]

On 12.11.2019. 10:54, Szél Gábor wrote:
> Dear Hrvoje, Theo,
> 
> Thank you for your answers!
> 
> answers to the questions:
> -  who is parent interface for carp?  -> vlan  ( carp10 interface parent
> vlan10 -> vlan10 interface  parent -> trunk0 )
> - why vlan interfaces don't have ip address ? -> it wasn't needed! i
> think vlan interface need only tag packages. Carp (over vlan) interface
> have IP address.

it's little strange to me to not have ip address on parent carp
interface, but if it works for you ... ok..

> - vether implies that you have bridge? -> yes whe have only one bridge
> for bridget openvpn clients, but  we will eliminate it.
> 
> 
> we will do the following:
> - refresh our backup firewall to oBSD 6.6
> - replace trunk interface with aggr
> - remove bridge interface

this is nice start to make you setup faster. big performance killer in
your setup is ipsec and old hardware. maybe oce(4) but i never tested
it, so i'm not sure ... if you can, change oce with ix, intel x520 is
not that expensive ..

bridge is slow, but only for traffic that goes through it. with ipsec,
the same second when tunnel is established, forwarding performance will
drop significantly on whole firewall ...

> if there was an update finised, I'll write again!

please do, i would like to hear