Re: msi_delroute() panic
I can try. Theo also sent me some ideas to try which involved cutting
out PCI Express hotplug support from
sys/dev/pci/ppb.c
Do you have reason to believe this has been fixed?
-Jim
On 1/11/13 8:25 AM, Sebastian Benoit wrote:
Jim Miller(jmil...@sri-inc.com) on 2013.01.09 15:20:03 -0500:
Hi All,
I saw a previous post back in late December where someone was discussing
a random panic() related to msi_delroute(). I'm having the same problem
I believe on a OpenBSD 5.1 box. The panic manifests itself about every
week but no I can't find any specific event that is the cause. It
appears related to the em driver thinking it is being removed. Any
ideas or suggestions on how to debug this further?
Can you reproduce it with a -current snapshot?
/Benno
ddb{2} trace
Debugger() at Debugger+0x5
panic() at panic+0xe4
msi_delroute() at msi_delroute+0x54
intr_disestablish() at intr_disestablish+0xe9
em_detach() at em_detach+0x26
config_detach() at config_detach+0x143
config_detach_children() at config_detach_children+0x3e
pci_detach_devices() at pci_detach_devices+0x15
ppb_hotplug_remove() at ppb_hotplug_remove+0x26
workq_thread() at workq_thread+0x33
end trace frame: 0x0, count: -10
ddb{2} show panic
msi_delroute: no msi capability
ddb{2} machine ddbcpu 1
Stopped at Debugger+0x5: leave
RUN AT LEAST 'trace' AND 'ps' AND INCLUDE OUTPUT WHEN REPORTING THIS PANIC!
IF RUNNING SMP, USE 'mach ddbcpu #' AND 'trace' ON OTHER PROCESSORS, TOO.
DO NOT EVEN BOTHER REPORTING THIS WITHOUT INCLUDING THAT INFORMATION!
ddb{1} ps
PID PPID PGRPUID S FLAGS WAIT COMMAND
7473 31910 7473 0 30x80 ttyin bash
21722 31910 21722 0 30x80 ttyin bash
31910 1 31910 0 30x80 selectscreen
5621 1 5621 0 30x80 ttyin getty
6818 1 6818 0 30x80 selectsendmail
27397 1 27397 0 30x80 ttyin getty
18642 1 18642 0 30x80 ttyin getty
20939 1 20939 0 30x80 ttyin getty
19043 1 19043 0 30x80 ttyin getty
22270 1 22270 0 30x80 ttyin getty
13738 1 13738 0 30x80 selectcron
17462 1 17462 99 30x80 poll sndiod
25578 1 25578 0 30x80 selectsshd
3068 6655 6655 68 30x80 selectsasyncd
6655 1 6655 0 30x80 selectsasyncd
8 4076 4076 68 30x80 selectisakmpd
4076 1 4076 0 30x80 netio isakmpd
16603 1 16603 0 30x80 poll ntpd
9965 14530 9965 83 30x80 poll ntpd
14530 1 14530 83 30x80 poll ntpd
28104 8325 8325 70 30x80 selectnamed
8325 1 8325 0 30x80 netio named
5060 11080 11080 74 30x80 bpf pflogd
11080 1 11080 0 30x80 netio pflogd
27911 10551 10551 73 30x80 poll syslogd
10551 1 10551 0 30x80 netio syslogd
16 0 0 0 30x100200 aiodoned aiodoned
15 0 0 0 30x100200 syncerupdate
14 0 0 0 30x100200 cleaner cleaner
13 0 0 0 30x100200 reaperreaper
12 0 0 0 30x100200 pgdaemon pagedaemon
11 0 0 0 30x100200 bored crypto
10 0 0 0 30x100200 pftm pfpurge
9 0 0 0 30x100200 usbtskusbtask
8 0 0 0 30x100200 usbatsk usbatsk
7 0 0 0 30x100200 acpi0 acpi0
6 0 0 0 7 0x40100200idle3
5 0 0 0 3 0x40100200idle2
*4 0 0 0 7 0x40100200idle1
3 0 0 0 70x100200syswq
2 0 0 0 7 0x40100200idle0
1 0 1 0 30x80 wait init
0 -1 0 0 3 0x200 scheduler swapper
ddb{1} machine ddbcpu 2
Stopped at Debugger+0x5: leave
RUN AT LEAST 'trace' AND 'ps' AND INCLUDE OUTPUT WHEN REPORTING THIS PANIC!
IF RUNNING SMP, USE 'mach ddbcpu #' AND 'trace' ON OTHER PROCESSORS, TOO.
DO NOT EVEN BOTHER REPORTING THIS WITHOUT INCLUDING THAT INFORMATION!
ddb{2} trace
Debugger() at Debugger+0x5
panic() at panic+0xe4
msi_delroute() at msi_delroute+0x54
intr_disestablish() at intr_disestablish+0xe9
em_detach() at em_detach+0x26
config_detach() at config_detach+0x143
config_detach_children
msi_delroute() panic
Hi All,
I saw a previous post back in late December where someone was discussing
a random panic() related to msi_delroute(). I'm having the same problem
I believe on a OpenBSD 5.1 box. The panic manifests itself about every
week but no I can't find any specific event that is the cause. It
appears related to the em driver thinking it is being removed. Any
ideas or suggestions on how to debug this further?
ddb{2} trace
Debugger() at Debugger+0x5
panic() at panic+0xe4
msi_delroute() at msi_delroute+0x54
intr_disestablish() at intr_disestablish+0xe9
em_detach() at em_detach+0x26
config_detach() at config_detach+0x143
config_detach_children() at config_detach_children+0x3e
pci_detach_devices() at pci_detach_devices+0x15
ppb_hotplug_remove() at ppb_hotplug_remove+0x26
workq_thread() at workq_thread+0x33
end trace frame: 0x0, count: -10
ddb{2} show panic
msi_delroute: no msi capability
ddb{2} machine ddbcpu 1
Stopped at Debugger+0x5: leave
RUN AT LEAST 'trace' AND 'ps' AND INCLUDE OUTPUT WHEN REPORTING THIS PANIC!
IF RUNNING SMP, USE 'mach ddbcpu #' AND 'trace' ON OTHER PROCESSORS, TOO.
DO NOT EVEN BOTHER REPORTING THIS WITHOUT INCLUDING THAT INFORMATION!
ddb{1} ps
PID PPID PGRPUID S FLAGS WAIT COMMAND
7473 31910 7473 0 30x80 ttyin bash
21722 31910 21722 0 30x80 ttyin bash
31910 1 31910 0 30x80 selectscreen
5621 1 5621 0 30x80 ttyin getty
6818 1 6818 0 30x80 selectsendmail
27397 1 27397 0 30x80 ttyin getty
18642 1 18642 0 30x80 ttyin getty
20939 1 20939 0 30x80 ttyin getty
19043 1 19043 0 30x80 ttyin getty
22270 1 22270 0 30x80 ttyin getty
13738 1 13738 0 30x80 selectcron
17462 1 17462 99 30x80 poll sndiod
25578 1 25578 0 30x80 selectsshd
3068 6655 6655 68 30x80 selectsasyncd
6655 1 6655 0 30x80 selectsasyncd
8 4076 4076 68 30x80 selectisakmpd
4076 1 4076 0 30x80 netio isakmpd
16603 1 16603 0 30x80 poll ntpd
9965 14530 9965 83 30x80 poll ntpd
14530 1 14530 83 30x80 poll ntpd
28104 8325 8325 70 30x80 selectnamed
8325 1 8325 0 30x80 netio named
5060 11080 11080 74 30x80 bpf pflogd
11080 1 11080 0 30x80 netio pflogd
27911 10551 10551 73 30x80 poll syslogd
10551 1 10551 0 30x80 netio syslogd
16 0 0 0 30x100200 aiodoned aiodoned
15 0 0 0 30x100200 syncerupdate
14 0 0 0 30x100200 cleaner cleaner
13 0 0 0 30x100200 reaperreaper
12 0 0 0 30x100200 pgdaemon pagedaemon
11 0 0 0 30x100200 bored crypto
10 0 0 0 30x100200 pftm pfpurge
9 0 0 0 30x100200 usbtskusbtask
8 0 0 0 30x100200 usbatsk usbatsk
7 0 0 0 30x100200 acpi0 acpi0
6 0 0 0 7 0x40100200idle3
5 0 0 0 3 0x40100200idle2
*4 0 0 0 7 0x40100200idle1
3 0 0 0 70x100200syswq
2 0 0 0 7 0x40100200idle0
1 0 1 0 30x80 wait init
0 -1 0 0 3 0x200 scheduler swapper
ddb{1} machine ddbcpu 2
Stopped at Debugger+0x5: leave
RUN AT LEAST 'trace' AND 'ps' AND INCLUDE OUTPUT WHEN REPORTING THIS PANIC!
IF RUNNING SMP, USE 'mach ddbcpu #' AND 'trace' ON OTHER PROCESSORS, TOO.
DO NOT EVEN BOTHER REPORTING THIS WITHOUT INCLUDING THAT INFORMATION!
ddb{2} trace
Debugger() at Debugger+0x5
panic() at panic+0xe4
msi_delroute() at msi_delroute+0x54
intr_disestablish() at intr_disestablish+0xe9
em_detach() at em_detach+0x26
config_detach() at config_detach+0x143
config_detach_children() at config_detach_children+0x3e
pci_detach_devices() at pci_detach_devices+0x15
ppb_hotplug_remove() at ppb_hotplug_remove+0x26
workq_thread() at workq_thread+0x33
end trace frame: 0x0, count: -10
ddb{2} machine ddbcpu 3
Stopped at Debugger+0x5: leave
RUN AT LEAST 'trace' AND 'ps' AND INCLUDE OUTPUT WHEN REPORTING THIS PANIC!
IF RUNNING SMP, USE 'mach ddbcpu #' AND 'trace' ON OTHER PROCESSORS, TOO.
DO NOT EVEN BOTHER
iked vs. isakmpd + carp
Two part question: 1. Anyone had any success getting iked and carp working on OpenBSD 5.1 (amd64)? We can get it working with isakmpd. The issue seems to be that iked wants to send out packets as the physical interface IP instead of the carp IP. iked documentation eludes to the fact that it should work. 2. I can't get isakmpd to use groups above modp1024 when using aes-256 or aes in main. Is there a catch I'm not aware of? Works: gwA = 1.1.1.1 gwB = 2.2.2.2 ike active esp from 192.168.1.1 to 172.16.1.1 \ local $gwA peer $gwB \ main auth hmac-sha1 enc aes-256 group modp1024 \ quick auth hmac-sha1 enc aes-256 \ psk foobar Does not work: gwA = 1.1.1.1 gwB = 2.2.2.2 ike active esp from 192.168.1.1 to 172.16.1.1 \ local $gwA peer $gwB \ main auth hmac-sha1 enc aes-256 group modp2048 \ quick auth hmac-sha1 enc aes-256 \ psk foobar The error message isakmpd spits out on one side is says MALFORMED_PAYLOAD and the other NO_PROPOSAL_CHOSEN. I can provide more details if needed. Just odd it works with only a change to the group field. thanks Jim
Re: iked vs. isakmpd + carp
Thanks Reky. I'll stick with isakmp for now but would like to swtich to iked when its ready. BTW. Any known issues with isakmp and groups larger than modp1024? I still can't get isakmpd to use anything larger than that? -Jim On 10/19/12 3:35 PM, Reyk Floeter wrote: Hi, On Fri, Oct 19, 2012 at 8:10 PM, Tyler Morgan tyl...@tradetech.net wrote: On 10/19/2012 1:16 AM, Jim Miller wrote: Two part question: 1. Anyone had any success getting iked and carp working on OpenBSD 5.1 (amd64)? We can get it working with isakmpd. The issue seems to be that iked wants to send out packets as the physical interface IP instead of the carp IP. iked documentation eludes to the fact that it should work. thanks for reporting, I can reproduce the problem. In my experience under 5.1 isakmpd wants to use the IP from the real physical interface instead of the virtual carp interface too, so I have to use the local x.x.x.x command in ipsec.conf, where x.x.x.x = my carp IP -- this forces it onto the carp IP and all is well. iked.conf(5) has a similar local command. Does it not work? It does not work. You can see that iked is setting the carp address correctly but the address on the wire is the primary one. Fail. The code doesn't bind() to the IP used in the local command and the kernel uses the primary address for the related route. btw. you can also specify local carp0 instead of the IP address and it will pick the interface's first address. and keep in mind the caveat: iked is not yet finished and is missing some important security features. It should not yet be used in production networks. -- iked(8) Yeah, but we're working on it. I actually added this comment before mikeb@ added support for SA expiration, lifetimes and retransmits. So iked is still not ready, but the situation is much better now ;-) reyk
Re: IPSEC VPN performance
I just reran the test again. I still receive about 600Mbps using iPerf however using client # dd if=/dev/zero bs=1000 count=100 | nc -v 172.16.2.2 12345 server # nc -v -l 12345 /dev/null I get numbers around 350Mbps. I tend to think iPerf is more reliable in this situation. Any ideas why the tests vary so much? -Jim On 9/28/12 9:18 PM, Ryan McBride wrote: 600Mbps seems about right, I tested a pair of E5649-based boxes to 550Mbps last year (with aes-128-gcm): http://marc.info/?l=openbsd-miscm=134033767126930 You'll probably get slightly more than 600 with with multiple TCP streams. Assuming PF was enabled for your test (the default configuration), the performance should be about the same with a proper ruleset. Traffic for existing states won't hit the ruleset at all. On Fri, Sep 28, 2012 at 06:39:14PM -0400, Jim Miller wrote: Yes. Let me double check everything again on Monday. Keep in mind that all devices had 1Gb ethernet interfaces and everything was directly cabled. No pf rules either. w/o ipsec I could get 900mbps through the openbsd boxes. Now you've got me thinking I need to recheck everything. -Jim On 9/28/12 5:19 PM, Hrvoje Popovski wrote: Hi, On 28.9.2012 22:09, Jim Miller wrote: So using another Mac w/ 1Gb ethernet adapter to a Linux box w/ 1Gb eth I was able to achieve approx. 600Mbps performance through the test setup (via iperf and my dd method). 600Mbps via ipsec between two Intel E31220 ?
Re: IPSEC VPN performance
PRO/1000 MT (82574L) rev 0x00: msi, address 00:25:90:75:91:c1 ppb3 at pci0 dev 28 function 6 Intel 6 Series PCIE rev 0xb5: apic 2 int 18 pci4 at ppb3 bus 4 em2 at pci4 dev 0 function 0 Intel PRO/1000 MT (82574L) rev 0x00: msi, address 00:25:90:75:91:c2 ppb4 at pci0 dev 28 function 7 Intel 6 Series PCIE rev 0xb5: apic 2 int 19 pci5 at ppb4 bus 5 em3 at pci5 dev 0 function 0 Intel PRO/1000 MT (82574L) rev 0x00: msi, address 00:25:90:75:91:c3 ehci1 at pci0 dev 29 function 0 Intel 6 Series USB rev 0x05: apic 2 int 23 usb1 at ehci1: USB revision 2.0 uhub1 at usb1 Intel EHCI root hub rev 2.00/1.00 addr 1 ppb5 at pci0 dev 30 function 0 Intel 82801BA Hub-to-PCI rev 0xa5 pci6 at ppb5 bus 6 vga1 at pci6 dev 3 function 0 Matrox MGA G200eW rev 0x0a wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) wsdisplay0: screen 1-5 added (80x25, vt100 emulation) pcib0 at pci0 dev 31 function 0 Intel C204 LPC rev 0x05 ahci0 at pci0 dev 31 function 2 Intel 6 Series AHCI rev 0x05: msi, AHCI 1.3 scsibus0 at ahci0: 32 targets sd0 at scsibus0 targ 0 lun 0: ATA, INTEL SSDSA2CT04, 4PC1 SCSI3 0/direct fixed naa.5001517972e60be6 sd0: 38166MB, 512 bytes/sector, 78165360 sectors, thin ichiic0 at pci0 dev 31 function 3 Intel 6 Series SMBus rev 0x05: apic 2 int 18 iic0 at ichiic0 sdtemp0 at iic0 addr 0x18: stts2002 sdtemp1 at iic0 addr 0x1a: stts2002 spdmem0 at iic0 addr 0x50: 1GB DDR3 SDRAM ECC PC3-10600 with thermal sensor spdmem1 at iic0 addr 0x52: 1GB DDR3 SDRAM ECC PC3-10600 with thermal sensor isa0 at pcib0 isadma0 at isa0 com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo com2 at isa0 port 0x3e8/8 irq 5: ns16550a, 16 byte fifo pckbc0 at isa0 port 0x60/5 pckbd0 at pckbc0 (kbd slot) pckbc0: using irq 1 for kbd slot wskbd0 at pckbd0: console keyboard, using wsdisplay0 pcppi0 at isa0 port 0x61 spkr0 at pcppi0 wbsio0 at isa0 port 0x2e/2: NCT6776F rev 0x33 lm1 at wbsio0 port 0xa30/8: NCT6776F npx0 at isa0 port 0xf0/16: reported by CPUID; using exception 16 mtrr: Pentium Pro MTRR support uhub2 at uhub0 port 1 Intel Rate Matching Hub rev 2.00/0.00 addr 2 uhidev0 at uhub2 port 2 configuration 1 interface 0 Winbond Electronics Corp Hermon USB hidmouse Device rev 1.10/0.01 addr 3 uhidev0: iclass 3/1 ums0 at uhidev0: 3 buttons, Z dir wsmouse0 at ums0 mux 0 uhidev1 at uhub2 port 2 configuration 1 interface 1 Winbond Electronics Corp Hermon USB hidmouse Device rev 1.10/0.01 addr 3 uhidev1: iclass 3/1 ukbd0 at uhidev1: 8 modifier keys, 6 key codes wskbd1 at ukbd0 mux 1 wskbd1: connecting to wsdisplay0 uhub3 at uhub1 port 1 Intel Rate Matching Hub rev 2.00/0.00 addr 2 uhidev2 at uhub3 port 2 configuration 1 interface 0 Generic USB Keyboard rev 1.10/2.05 addr 3 uhidev2: iclass 3/1 ukbd1 at uhidev2: 8 modifier keys, 6 key codes wskbd2 at ukbd1 mux 1 wskbd2: connecting to wsdisplay0 uhidev3 at uhub3 port 2 configuration 1 interface 1 Generic USB Keyboard rev 1.10/2.05 addr 3 uhidev3: iclass 3/1, 3 report ids uhid0 at uhidev3 reportid 1: input=1, output=0, feature=0 uhid1 at uhidev3 reportid 2: input=3, output=0, feature=0 ums1 at uhidev3 reportid 3: 0 button, Z dir wsmouse1 at ums1 mux 0 vscsi0 at root scsibus1 at vscsi0: 256 targets softraid0 at root scsibus2 at softraid0: 256 targets root on sd0a (532543e2c86874e1.a) swap on sd0b dump on sd0b On 9/28/12 6:28 AM, Mike Belopuhov wrote: On Fri, Sep 28, 2012 at 11:45 AM, Otto Moerbeek o...@drijf.net wrote: On Thu, Sep 27, 2012 at 05:30:38PM -0400, Jim Miller wrote: Hi, I'm trying to determine if the performance I'm seeing between two OpenBSD 5.1 IPSEC VPN endpoints is typical (or expected). I recognize there are quite a few variables to consider and I'm sure I've not toggled each one but I could use a sanity check regardless. Question: With the configuration below when I disable ipsec I can route traffic between the two hosts (hosts A and B) at about 900mbps. When I add the VPN I am getting speeds of approx. 40mbps. The CPU load on the OpenBSD boxes spikes to about 80% on one of the cores but the other 3 are essentially unaffected. Enabling/Disabling AES-NI in the bios doesn't seem to actually do anything as the cpu message in dmesg still shows the AES flag. The test I'm using is this Host A: # nc -v -l 12345 | /dev/null Host B: # dd if=/dev/zero bs=1000 count=1 | nc -v host a 12345 The reason these performance numbers are concerning to me is that I wanted a solution that would allow me to get decent (a.k.a. 100mbps +/- 10%) without having to buy expensive cisco/juniper devices. I would start playing with different modes, to see if that makes a difference. It could very well be that AES-NI is only used in certain modes. Start with the iked defaults for a start. aes-ni is used for all aes-related modes (aes-cbc, aes-ctr, aes-gcm and aes-gmac)... on amd64. Am I dreaming or have others had better performance? Also, any recent data on AES-NI optimizations would be helpful. Thanks Jim
Re: IPSEC VPN performance
pci1 at ppb0 bus 1 ppb1 at pci0 dev 28 function 4 Intel 6 Series PCIE rev 0xb5: msi pci2 at ppb1 bus 2 em0 at pci2 dev 0 function 0 Intel PRO/1000 MT (82574L) rev 0x00: msi, address 00:25:90:75:91:c0 ppb2 at pci0 dev 28 function 5 Intel 6 Series PCIE rev 0xb5: msi pci3 at ppb2 bus 3 em1 at pci3 dev 0 function 0 Intel PRO/1000 MT (82574L) rev 0x00: msi, address 00:25:90:75:91:c1 ppb3 at pci0 dev 28 function 6 Intel 6 Series PCIE rev 0xb5: msi pci4 at ppb3 bus 4 em2 at pci4 dev 0 function 0 Intel PRO/1000 MT (82574L) rev 0x00: msi, address 00:25:90:75:91:c2 ppb4 at pci0 dev 28 function 7 Intel 6 Series PCIE rev 0xb5: msi pci5 at ppb4 bus 5 em3 at pci5 dev 0 function 0 Intel PRO/1000 MT (82574L) rev 0x00: msi, address 00:25:90:75:91:c3 ehci1 at pci0 dev 29 function 0 Intel 6 Series USB rev 0x05: apic 2 int 23 usb1 at ehci1: USB revision 2.0 uhub1 at usb1 Intel EHCI root hub rev 2.00/1.00 addr 1 ppb5 at pci0 dev 30 function 0 Intel 82801BA Hub-to-PCI rev 0xa5 pci6 at ppb5 bus 6 vga1 at pci6 dev 3 function 0 Matrox MGA G200eW rev 0x0a wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) wsdisplay0: screen 1-5 added (80x25, vt100 emulation) pcib0 at pci0 dev 31 function 0 Intel C204 LPC rev 0x05 ahci0 at pci0 dev 31 function 2 Intel 6 Series AHCI rev 0x05: msi, AHCI 1.3 scsibus0 at ahci0: 32 targets sd0 at scsibus0 targ 0 lun 0: ATA, INTEL SSDSA2CT04, 4PC1 SCSI3 0/direct fixed naa.5001517972e60be6 sd0: 38166MB, 512 bytes/sector, 78165360 sectors, thin ichiic0 at pci0 dev 31 function 3 Intel 6 Series SMBus rev 0x05: apic 2 int 18 iic0 at ichiic0 sdtemp0 at iic0 addr 0x18: stts2002 sdtemp1 at iic0 addr 0x1a: stts2002 spdmem0 at iic0 addr 0x50: 1GB DDR3 SDRAM ECC PC3-10600 with thermal sensor spdmem1 at iic0 addr 0x52: 1GB DDR3 SDRAM ECC PC3-10600 with thermal sensor isa0 at pcib0 isadma0 at isa0 com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo pckbc0 at isa0 port 0x60/5 pckbd0 at pckbc0 (kbd slot) pckbc0: using irq 1 for kbd slot wskbd0 at pckbd0: console keyboard, using wsdisplay0 pcppi0 at isa0 port 0x61 spkr0 at pcppi0 wbsio0 at isa0 port 0x2e/2: NCT6776F rev 0x33 lm1 at wbsio0 port 0xa30/8: NCT6776F mtrr: Pentium Pro MTRR support uhub2 at uhub0 port 1 Intel Rate Matching Hub rev 2.00/0.00 addr 2 uhidev0 at uhub2 port 2 configuration 1 interface 0 Winbond Electronics Corp Hermon USB hidmouse Device rev 1.10/0.01 addr 3 uhidev0: iclass 3/1 ums0 at uhidev0: 3 buttons, Z dir wsmouse0 at ums0 mux 0 uhidev1 at uhub2 port 2 configuration 1 interface 1 Winbond Electronics Corp Hermon USB hidmouse Device rev 1.10/0.01 addr 3 uhidev1: iclass 3/1 ukbd0 at uhidev1: 8 modifier keys, 6 key codes wskbd1 at ukbd0 mux 1 wskbd1: connecting to wsdisplay0 uhub3 at uhub1 port 1 Intel Rate Matching Hub rev 2.00/0.00 addr 2 uhidev2 at uhub3 port 2 configuration 1 interface 0 Generic USB Keyboard rev 1.10/2.05 addr 3 uhidev2: iclass 3/1 ukbd1 at uhidev2: 8 modifier keys, 6 key codes wskbd2 at ukbd1 mux 1 wskbd2: connecting to wsdisplay0 uhidev3 at uhub3 port 2 configuration 1 interface 1 Generic USB Keyboard rev 1.10/2.05 addr 3 uhidev3: iclass 3/1, 3 report ids uhid0 at uhidev3 reportid 1: input=1, output=0, feature=0 uhid1 at uhidev3 reportid 2: input=3, output=0, feature=0 ums1 at uhidev3 reportid 3: 0 button, Z dir wsmouse1 at ums1 mux 0 vscsi0 at root scsibus1 at vscsi0: 256 targets softraid0 at root scsibus2 at softraid0: 256 targets root on sd0a (124c86b6eb046834.a) swap on sd0b dump on sd0b On 9/28/12 8:40 AM, Otto Moerbeek wrote: On Fri, Sep 28, 2012 at 08:38:37AM -0400, Jim Miller wrote: Sorry I was stingy on the dmesg output. Here's the full dump. I will test with other AES modes now. And then install amd64 ;-) -Otto -Jim OpenBSD 5.1 (GENERIC.MP) #188: Sun Feb 12 09:55:11 MST 2012 dera...@i386.openbsd.org:/usr/src/sys/arch/i386/compile/GENERIC.MP cpu0: Intel(R) Xeon(R) CPU E31220 @ 3.10GHz (GenuineIntel 686-class) 3.10 GHz cpu0: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,NXE,LONG,SSE3,PCLMUL,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,x2APIC,POPCNT,AES,XSAVE,AVX,LAHF real mem = 2119032832 (2020MB) avail mem = 2074247168 (1978MB) mainbus0 at root bios0 at mainbus0: AT/286+ BIOS, date 12/22/11, SMBIOS rev. 2.7 @ 0xeb4c0 (54 entries) bios0: vendor American Megatrends Inc. version 2.00 date 05/08/2012 bios0: Supermicro X9SCI/X9SCA acpi0 at bios0: rev 2 acpi0: sleep states S0 S1 S4 S5 acpi0: tables DSDT FACP APIC FPDT MCFG PRAD HPET SSDT SPMI SSDT SSDT SPCR EINJ ERST HEST BERT acpi0: wakeup devices PS2K(S4) PS2M(S4) UAR1(S4) UAR2(S4) P0P1(S4) USB1(S4) USB2(S4) USB3(S4) USB4(S4) USB5(S4) USB6(S4) USB7(S4) PXSX(S4) RP01(S4) PXSX(S4) RP02(S4) PXSX(S4) RP03(S4) PXSX(S4) RP04(S4) PXSX(S4) RP05(S4) PXSX(S4) RP06(S4) PXSX(S4) RP07(S4) PXSX(S4) RP08(S4) PEGP(S4) PEG0(S4) PEG1(S4) PEG2(S4) PEG3(S4) GLAN(S4) EHC1(S4) EHC2
Re: IPSEC VPN performance
So I just realized another serious flaw in my testing. I was using a Mac Air w/ USB 100Mb ethernet adapter for one of the hosts behind the OpenBSD VPN devices. And it must have been limiting the speed more than I thought. So using another Mac w/ 1Gb ethernet adapter to a Linux box w/ 1Gb eth I was able to achieve approx. 600Mbps performance through the test setup (via iperf and my dd method). Still it baffles me as to why the ASA 5505 performed better with the Mac Air's USB 100mbps connection than the OpenVPN boxes. The ASA was able to do approx 88mbps while I never got above 72mbps on the OpenBSD test. Either way, case closed. I'd say that's fast enough. Lessons' learned: - Use the amd64 kernel not i386 - w/ AES-NI enabled AES-256-GMAC, AES-256-GCM, AES-128 all performed about the same - For some reason on my supermicro board disabling AES-NI doesn't have an effect as OpenBSD still seems to find the instructions - Don't use USB for testing performance. ;) Thanks to all that helped. -Jim On 9/28/12 3:10 PM, Christian Weisgerber wrote: Jim Miller jmil...@sri-inc.com wrote: The test I'm using is this Host A: # nc -v -l 12345 | /dev/null Host B: # dd if=/dev/zero bs=1000 count=1 | nc -v host a 12345 I increased the count a bit: 10 bytes transferred in 53.265 secs (18773882 bytes/sec) That's with AES-256-GCM between two Sandy Bridge Xeons (Intel Xeon CPU E5-2637 @ 3.00GHz), i.e., with AES-NI, running OpenBSD-current/amd64.
Re: IPSEC VPN performance
Yes. Let me double check everything again on Monday. Keep in mind that all devices had 1Gb ethernet interfaces and everything was directly cabled. No pf rules either. w/o ipsec I could get 900mbps through the openbsd boxes. Now you've got me thinking I need to recheck everything. -Jim On 9/28/12 5:19 PM, Hrvoje Popovski wrote: Hi, On 28.9.2012 22:09, Jim Miller wrote: So using another Mac w/ 1Gb ethernet adapter to a Linux box w/ 1Gb eth I was able to achieve approx. 600Mbps performance through the test setup (via iperf and my dd method). 600Mbps via ipsec between two Intel E31220 ?
IPSEC VPN performance
Hi, I'm trying to determine if the performance I'm seeing between two OpenBSD 5.1 IPSEC VPN endpoints is typical (or expected). I recognize there are quite a few variables to consider and I'm sure I've not toggled each one but I could use a sanity check regardless. Question: With the configuration below when I disable ipsec I can route traffic between the two hosts (hosts A and B) at about 900mbps. When I add the VPN I am getting speeds of approx. 40mbps. The CPU load on the OpenBSD boxes spikes to about 80% on one of the cores but the other 3 are essentially unaffected. Enabling/Disabling AES-NI in the bios doesn't seem to actually do anything as the cpu message in dmesg still shows the AES flag. The test I'm using is this Host A: # nc -v -l 12345 | /dev/null Host B: # dd if=/dev/zero bs=1000 count=1 | nc -v host a 12345 The reason these performance numbers are concerning to me is that I wanted a solution that would allow me to get decent (a.k.a. 100mbps +/- 10%) without having to buy expensive cisco/juniper devices. Am I dreaming or have others had better performance? Also, any recent data on AES-NI optimizations would be helpful. Thanks Jim Hardware Configuration: - (2) identical SuperMicro systems with quad core E31220 w/ AES-NI enabled cpu0: Intel(R) Xeon(R) CPU E31220 @ 3.10GHz (GenuineIntel 686-class) 3.10 GHz cpu0: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,NXE,LONG,SSE3,PCLMUL,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,x2APIC,POPCNT,AES,XSAVE,AVX,LAHF cpu1: .. cpu2: ... cpu3: ... - 2GB ram - AES-NI enabled in bios - (4) Intel PRO/1000 MT (82574L) Software Configuration: VPN A /etc/iked.conf ikev2 active esp \ from 172.16.1.0/24 to 172.16.2.0/24 \ local 10.0.0.1 peer 10.0.0.2 \ ikesa enc aes-256 auth hmac-sha2-512 group modp4096 \ childsa enc aes-256-gmac \ psk helpmeplease VPN B (reverse of A config) Host A - 172.16.1.2 (behind VPN A) Host B- 172.16.2.2 (behind VPN B) VPN A (10.0.0.1) talks to B (10.0.0.2) via a crossover cable. No switches/routers/hubs/etc in this test system. All hosts running linux with 1000mb phys.
