Re: Packet loss with latest snapshot

[Tony Sarendal]

On Mon, 4 Mar 2019, 13:29 David Gwynne,  wrote:

> On Mon, Mar 04, 2019 at 10:36:23AM +0100, Tony Sarendal wrote:
> > On Mon, 4 Mar 2019, 09:43 Tony Sarendal,  wrote:
> >
> > >
> > >
> > > Den m??n 4 mars 2019 kl 09:26 skrev Tony Sarendal :
> > >
> > >> Den s??n 3 mars 2019 kl 21:35 skrev Theo de Raadt <
> dera...@openbsd.org>:
> > >>
> > >>> Tony,
> > >>>
> > >>> Are you out of your mind?  You didn't provide even a rough hint about
> > >>> what your firewall configuration looks like.  You recognize that's
> > >>> pathetic, right?
> > >>>
> > >>> > Earlier in the week I could run parallel ping-pong tests through my
> > >>> test
> > >>> > firewalls
> > >>> > at 300kpps without any packet loss. I updated to the latest
> snapshot
> > >>> today
> > >>> > and
> > >>> > start to see packet loss at around 80kpps.
> > >>> >
> > >>> > /T
> > >>> >
> > >>> > OpenBSD 6.5-beta (GENERIC.MP) #764: Sun Mar  3 10:24:08 MST 2019
> > >>> > dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/
> > >>> GENERIC.MP
> > >>> > real mem = 34300891136 (32711MB)
> > >>> > avail mem = 33251393536 (31711MB)
> > >>> > mpath0 at root
> > >>> > scsibus0 at mpath0: 256 targets
> > >>> > mainbus0 at root
> > >>> > bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xec170 (34 entries)
> > >>> > bios0: vendor American Megatrends Inc. version "3.0" date
> 04/24/2015
> > >>> > bios0: Supermicro X10SLD
> > >>> > acpi0 at bios0: rev 2
> > >>> > acpi0: sleep states S0 S4 S5
> > >>> > acpi0: tables DSDT FACP APIC FPDT FIDT SSDT SSDT MCFG PRAD HPET
> SSDT
> > >>> SSDT
> > >>> > SPMI DMAR EINJ ERST HEST BERT
> > >>> > acpi0: wakeup devices PEGP(S4) PEG0(S4) PEGP(S4) PEG1(S4) PEGP(S4)
> > >>> PEG2(S4)
> > >>> > PXSX(S4) RP01(S4) PXSX(S4) RP02(S4) PXSX(S4) RP03(S4) PXSX(S4)
> RP04(S4)
> > >>> > PXSX(S4) RP05(S4) [...]
> > >>> > acpitimer0 at acpi0: 3579545 Hz, 24 bits
> > >>> > acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
> > >>> > cpu0 at mainbus0: apid 0 (boot processor)
> > >>> > cpu0: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.68 MHz,
> 06-3c-03
> > >>> > cpu0:
> > >>> >
> > >>>
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> > >>> > cpu0: 256KB 64b/line 8-way L2 cache
> > >>> > cpu0: smt 0, core 0, package 0
> > >>> > mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
> > >>> > cpu0: apic clock running at 99MHz
> > >>> > cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4, IBE
> > >>> > cpu1 at mainbus0: apid 2 (application processor)
> > >>> > cpu1: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.01 MHz,
> 06-3c-03
> > >>> > cpu1:
> > >>> >
> > >>>
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> > >>> > cpu1: 256KB 64b/line 8-way L2 cache
> > >>> > cpu1: smt 0, core 1, package 0
> > >>> > cpu2 at mainbus0: apid 4 (application processor)
> > >>> > cpu2: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.01 MHz,
> 06-3c-03
> > >>> > cpu2:
> > >>> >
> > >>>
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE

Re: Packet loss with latest snapshot

[Tony Sarendal]

On Mon, 4 Mar 2019, 09:43 Tony Sarendal,  wrote:

>
>
> Den mån 4 mars 2019 kl 09:26 skrev Tony Sarendal :
>
>> Den sön 3 mars 2019 kl 21:35 skrev Theo de Raadt :
>>
>>> Tony,
>>>
>>> Are you out of your mind?  You didn't provide even a rough hint about
>>> what your firewall configuration looks like.  You recognize that's
>>> pathetic, right?
>>>
>>> > Earlier in the week I could run parallel ping-pong tests through my
>>> test
>>> > firewalls
>>> > at 300kpps without any packet loss. I updated to the latest snapshot
>>> today
>>> > and
>>> > start to see packet loss at around 80kpps.
>>> >
>>> > /T
>>> >
>>> > OpenBSD 6.5-beta (GENERIC.MP) #764: Sun Mar  3 10:24:08 MST 2019
>>> > dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/
>>> GENERIC.MP
>>> > real mem = 34300891136 (32711MB)
>>> > avail mem = 33251393536 (31711MB)
>>> > mpath0 at root
>>> > scsibus0 at mpath0: 256 targets
>>> > mainbus0 at root
>>> > bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xec170 (34 entries)
>>> > bios0: vendor American Megatrends Inc. version "3.0" date 04/24/2015
>>> > bios0: Supermicro X10SLD
>>> > acpi0 at bios0: rev 2
>>> > acpi0: sleep states S0 S4 S5
>>> > acpi0: tables DSDT FACP APIC FPDT FIDT SSDT SSDT MCFG PRAD HPET SSDT
>>> SSDT
>>> > SPMI DMAR EINJ ERST HEST BERT
>>> > acpi0: wakeup devices PEGP(S4) PEG0(S4) PEGP(S4) PEG1(S4) PEGP(S4)
>>> PEG2(S4)
>>> > PXSX(S4) RP01(S4) PXSX(S4) RP02(S4) PXSX(S4) RP03(S4) PXSX(S4) RP04(S4)
>>> > PXSX(S4) RP05(S4) [...]
>>> > acpitimer0 at acpi0: 3579545 Hz, 24 bits
>>> > acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
>>> > cpu0 at mainbus0: apid 0 (boot processor)
>>> > cpu0: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.68 MHz, 06-3c-03
>>> > cpu0:
>>> >
>>> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
>>> > cpu0: 256KB 64b/line 8-way L2 cache
>>> > cpu0: smt 0, core 0, package 0
>>> > mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
>>> > cpu0: apic clock running at 99MHz
>>> > cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4, IBE
>>> > cpu1 at mainbus0: apid 2 (application processor)
>>> > cpu1: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.01 MHz, 06-3c-03
>>> > cpu1:
>>> >
>>> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
>>> > cpu1: 256KB 64b/line 8-way L2 cache
>>> > cpu1: smt 0, core 1, package 0
>>> > cpu2 at mainbus0: apid 4 (application processor)
>>> > cpu2: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.01 MHz, 06-3c-03
>>> > cpu2:
>>> >
>>> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
>>> > cpu2: 256KB 64b/line 8-way L2 cache
>>> > cpu2: smt 0, core 2, package 0
>>> > cpu3 at mainbus0: apid 6 (application processor)
>>> > cpu3: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.01 MHz, 06-3c-03
>>> > cpu3:
>>> >
>>> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,

Re: Packet loss with latest snapshot

[Tony Sarendal]

Den mån 4 mars 2019 kl 09:26 skrev Tony Sarendal :

> Den sön 3 mars 2019 kl 21:35 skrev Theo de Raadt :
>
>> Tony,
>>
>> Are you out of your mind?  You didn't provide even a rough hint about
>> what your firewall configuration looks like.  You recognize that's
>> pathetic, right?
>>
>> > Earlier in the week I could run parallel ping-pong tests through my test
>> > firewalls
>> > at 300kpps without any packet loss. I updated to the latest snapshot
>> today
>> > and
>> > start to see packet loss at around 80kpps.
>> >
>> > /T
>> >
>> > OpenBSD 6.5-beta (GENERIC.MP) #764: Sun Mar  3 10:24:08 MST 2019
>> > dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/
>> GENERIC.MP
>> > real mem = 34300891136 (32711MB)
>> > avail mem = 33251393536 (31711MB)
>> > mpath0 at root
>> > scsibus0 at mpath0: 256 targets
>> > mainbus0 at root
>> > bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xec170 (34 entries)
>> > bios0: vendor American Megatrends Inc. version "3.0" date 04/24/2015
>> > bios0: Supermicro X10SLD
>> > acpi0 at bios0: rev 2
>> > acpi0: sleep states S0 S4 S5
>> > acpi0: tables DSDT FACP APIC FPDT FIDT SSDT SSDT MCFG PRAD HPET SSDT
>> SSDT
>> > SPMI DMAR EINJ ERST HEST BERT
>> > acpi0: wakeup devices PEGP(S4) PEG0(S4) PEGP(S4) PEG1(S4) PEGP(S4)
>> PEG2(S4)
>> > PXSX(S4) RP01(S4) PXSX(S4) RP02(S4) PXSX(S4) RP03(S4) PXSX(S4) RP04(S4)
>> > PXSX(S4) RP05(S4) [...]
>> > acpitimer0 at acpi0: 3579545 Hz, 24 bits
>> > acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
>> > cpu0 at mainbus0: apid 0 (boot processor)
>> > cpu0: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.68 MHz, 06-3c-03
>> > cpu0:
>> >
>> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
>> > cpu0: 256KB 64b/line 8-way L2 cache
>> > cpu0: smt 0, core 0, package 0
>> > mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
>> > cpu0: apic clock running at 99MHz
>> > cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4, IBE
>> > cpu1 at mainbus0: apid 2 (application processor)
>> > cpu1: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.01 MHz, 06-3c-03
>> > cpu1:
>> >
>> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
>> > cpu1: 256KB 64b/line 8-way L2 cache
>> > cpu1: smt 0, core 1, package 0
>> > cpu2 at mainbus0: apid 4 (application processor)
>> > cpu2: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.01 MHz, 06-3c-03
>> > cpu2:
>> >
>> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
>> > cpu2: 256KB 64b/line 8-way L2 cache
>> > cpu2: smt 0, core 2, package 0
>> > cpu3 at mainbus0: apid 6 (application processor)
>> > cpu3: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.01 MHz, 06-3c-03
>> > cpu3:
>> >
>> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
>> > cpu3: 256KB 64b/line 8-way L2 cache
>> > cpu3: smt 0, core 3, package 0
>> > ioapic0 at mainbus0: apid 8 pa 0xfec0, version 20, 24 pins
>> > acpimcfg0 at acpi0
>> > acpimcfg0: addr 0xf800, bus 0-63
>> > acpihpet0 at acpi0: 14318179 Hz
>>

Re: Packet loss with latest snapshot

[Tony Sarendal]

Den sön 3 mars 2019 kl 21:35 skrev Theo de Raadt :

> Tony,
>
> Are you out of your mind?  You didn't provide even a rough hint about
> what your firewall configuration looks like.  You recognize that's
> pathetic, right?
>
> > Earlier in the week I could run parallel ping-pong tests through my test
> > firewalls
> > at 300kpps without any packet loss. I updated to the latest snapshot
> today
> > and
> > start to see packet loss at around 80kpps.
> >
> > /T
> >
> > OpenBSD 6.5-beta (GENERIC.MP) #764: Sun Mar  3 10:24:08 MST 2019
> > dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
> > real mem = 34300891136 (32711MB)
> > avail mem = 33251393536 (31711MB)
> > mpath0 at root
> > scsibus0 at mpath0: 256 targets
> > mainbus0 at root
> > bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xec170 (34 entries)
> > bios0: vendor American Megatrends Inc. version "3.0" date 04/24/2015
> > bios0: Supermicro X10SLD
> > acpi0 at bios0: rev 2
> > acpi0: sleep states S0 S4 S5
> > acpi0: tables DSDT FACP APIC FPDT FIDT SSDT SSDT MCFG PRAD HPET SSDT SSDT
> > SPMI DMAR EINJ ERST HEST BERT
> > acpi0: wakeup devices PEGP(S4) PEG0(S4) PEGP(S4) PEG1(S4) PEGP(S4)
> PEG2(S4)
> > PXSX(S4) RP01(S4) PXSX(S4) RP02(S4) PXSX(S4) RP03(S4) PXSX(S4) RP04(S4)
> > PXSX(S4) RP05(S4) [...]
> > acpitimer0 at acpi0: 3579545 Hz, 24 bits
> > acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
> > cpu0 at mainbus0: apid 0 (boot processor)
> > cpu0: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.68 MHz, 06-3c-03
> > cpu0:
> >
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> > cpu0: 256KB 64b/line 8-way L2 cache
> > cpu0: smt 0, core 0, package 0
> > mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
> > cpu0: apic clock running at 99MHz
> > cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4, IBE
> > cpu1 at mainbus0: apid 2 (application processor)
> > cpu1: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.01 MHz, 06-3c-03
> > cpu1:
> >
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> > cpu1: 256KB 64b/line 8-way L2 cache
> > cpu1: smt 0, core 1, package 0
> > cpu2 at mainbus0: apid 4 (application processor)
> > cpu2: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.01 MHz, 06-3c-03
> > cpu2:
> >
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> > cpu2: 256KB 64b/line 8-way L2 cache
> > cpu2: smt 0, core 2, package 0
> > cpu3 at mainbus0: apid 6 (application processor)
> > cpu3: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.01 MHz, 06-3c-03
> > cpu3:
> >
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
> > cpu3: 256KB 64b/line 8-way L2 cache
> > cpu3: smt 0, core 3, package 0
> > ioapic0 at mainbus0: apid 8 pa 0xfec0, version 20, 24 pins
> > acpimcfg0 at acpi0
> > acpimcfg0: addr 0xf800, bus 0-63
> > acpihpet0 at acpi0: 14318179 Hz
> > acpiprt0 at acpi0: bus 0 (PCI0)
> > acpiprt1 at acpi0: bus 1 (PEG0)
> > acpiprt2 at acpi0: bus 2 (PEG1)
> > acpiprt3 at acpi0: bus -1 (PEG2)
> > acpiprt4 at acpi0: bus 3 (RP01)
> > acpiprt5 at acpi0: bus -1 (RP02)
> > acpiprt6 at acpi0: bus -1 (RP03)
> > acpiprt7 at acpi0: bus -1 (RP04)
> > acpiprt8 at acpi0: bus -1 (RP05)
> > acpiprt9 at acpi0: bus -1 (RP06)
> > acpiprt10 at acpi0: bus -1 (RP07)
> > acpiprt11 at acpi0: bus -1 (RP08)
> > acpiec0 at acpi0: not present
> > acpicpu0 at acpi0: C1(@1 halt!)
> > acpicpu1 at acpi0: C1(@1 halt!)
> > acpicpu2 at acpi0: C1(@1 halt!)
> > acpicpu3 at acpi0: C1(@1 halt!)
> > acpipwrres0 at acpi0: PG00, resource for PEG0
> > acpipwrres1 at acpi0: PG01, resource for PEG1
> > acpipwrres2 at acpi0: PG02, resource for 

Packet loss with latest snapshot

[Tony Sarendal]

Earlier in the week I could run parallel ping-pong tests through my test
firewalls
at 300kpps without any packet loss. I updated to the latest snapshot today
and
start to see packet loss at around 80kpps.

/T

OpenBSD 6.5-beta (GENERIC.MP) #764: Sun Mar  3 10:24:08 MST 2019
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 34300891136 (32711MB)
avail mem = 33251393536 (31711MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xec170 (34 entries)
bios0: vendor American Megatrends Inc. version "3.0" date 04/24/2015
bios0: Supermicro X10SLD
acpi0 at bios0: rev 2
acpi0: sleep states S0 S4 S5
acpi0: tables DSDT FACP APIC FPDT FIDT SSDT SSDT MCFG PRAD HPET SSDT SSDT
SPMI DMAR EINJ ERST HEST BERT
acpi0: wakeup devices PEGP(S4) PEG0(S4) PEGP(S4) PEG1(S4) PEGP(S4) PEG2(S4)
PXSX(S4) RP01(S4) PXSX(S4) RP02(S4) PXSX(S4) RP03(S4) PXSX(S4) RP04(S4)
PXSX(S4) RP05(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.68 MHz, 06-3c-03
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.01 MHz, 06-3c-03
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.01 MHz, 06-3c-03
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 6 (application processor)
cpu3: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.01 MHz, 06-3c-03
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu3: 256KB 64b/line 8-way L2 cache
cpu3: smt 0, core 3, package 0
ioapic0 at mainbus0: apid 8 pa 0xfec0, version 20, 24 pins
acpimcfg0 at acpi0
acpimcfg0: addr 0xf800, bus 0-63
acpihpet0 at acpi0: 14318179 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 1 (PEG0)
acpiprt2 at acpi0: bus 2 (PEG1)
acpiprt3 at acpi0: bus -1 (PEG2)
acpiprt4 at acpi0: bus 3 (RP01)
acpiprt5 at acpi0: bus -1 (RP02)
acpiprt6 at acpi0: bus -1 (RP03)
acpiprt7 at acpi0: bus -1 (RP04)
acpiprt8 at acpi0: bus -1 (RP05)
acpiprt9 at acpi0: bus -1 (RP06)
acpiprt10 at acpi0: bus -1 (RP07)
acpiprt11 at acpi0: bus -1 (RP08)
acpiec0 at acpi0: not present
acpicpu0 at acpi0: C1(@1 halt!)
acpicpu1 at acpi0: C1(@1 halt!)
acpicpu2 at acpi0: C1(@1 halt!)
acpicpu3 at acpi0: C1(@1 halt!)
acpipwrres0 at acpi0: PG00, resource for PEG0
acpipwrres1 at acpi0: PG01, resource for PEG1
acpipwrres2 at acpi0: PG02, resource for PEG2
acpipwrres3 at acpi0: FN00, resource for FAN0
acpipwrres4 at acpi0: FN01, resource for FAN1
acpipwrres5 at acpi0: FN02, resource for FAN2
acpipwrres6 at acpi0: FN03, resource for FAN3
acpipwrres7 at acpi0: FN04, resource for FAN4
acpitz0 at acpi0: critical temperature is 105 degC
acpitz1 at acpi0: critical temperature is 105 degC
acpipci0 at acpi0 PCI0: 0x 0x0011 0x0001
acpicmos0 at acpi0
"IPI0001" at acpi0 not configured
acpibtn0 at acpi0: SLPB
acpibtn1 at acpi0: PWRB
"PNP0C0B" at acpi0 not configured
"PNP0C0B" at 

40G ixl nics

[Tony Sarendal]

Good evening,

We inserted a 2x40G NIC into one of our old franken-pc's, and got this:

ixl0 at pci2 dev 0 function 0 "Intel XL710 QSFP+" rev 0x02: port 0, FW
5.0.40043 API 1.5, msi, address 0c:c4:7a:5e:f9:c8
ixl0: unable to query phy types
ixl1 at pci2 dev 0 function 1 "Intel XL710 QSFP+" rev 0x02: port 1, FW
5.0.40043 API 1.5, msi, address 0c:c4:7a:5e:f9:c9
ixl1: unable to query phy types

NIC:
https://www.supermicro.com/manuals/other/datasheet-AOC-S40G-i1Q_i2Q.pdf

Any ideas ?

Regards Tony

OpenBSD 6.4-current (GENERIC.MP) #658: Fri Feb  1 02:25:34 MST 2019
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 34300891136 (32711MB)
avail mem = 33251758080 (31711MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xec170 (34 entries)
bios0: vendor American Megatrends Inc. version "3.0" date 04/24/2015
bios0: Supermicro X10SLD
acpi0 at bios0: rev 2
acpi0: sleep states S0 S4 S5
acpi0: tables DSDT FACP APIC FPDT FIDT SSDT SSDT MCFG PRAD HPET SSDT SSDT
SPMI DMAR EINJ ERST HEST BERT
acpi0: wakeup devices PEGP(S4) PEG0(S4) PEGP(S4) PEG1(S4) PEGP(S4) PEG2(S4)
PXSX(S4) RP01(S4) PXSX(S4) RP02(S4) PXSX(S4) RP03(S4) PXSX(S4) RP04(S4)
PXSX(S4) RP05(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.69 MHz, 06-3c-03
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LO
NG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 100MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.01 MHz, 06-3c-03
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LO
NG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.01 MHz, 06-3c-03
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LO
NG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 6 (application processor)
cpu3: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.01 MHz, 06-3c-03
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LO
NG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu3: 256KB 64b/line 8-way L2 cache
cpu3: smt 0, core 3, package 0
ioapic0 at mainbus0: apid 8 pa 0xfec0, version 20, 24 pins
acpimcfg0 at acpi0
acpimcfg0: addr 0xf800, bus 0-63
acpihpet0 at acpi0: 14318179 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 1 (PEG0)
acpiprt2 at acpi0: bus 2 (PEG1)
acpiprt3 at acpi0: bus -1 (PEG2)
acpiprt4 at acpi0: bus 3 (RP01)
acpiprt5 at acpi0: bus -1 (RP02)
acpiprt6 at acpi0: bus -1 (RP03)
acpiprt7 at acpi0: bus -1 (RP04)
acpiprt8 at acpi0: bus -1 (RP05)
acpiprt9 at acpi0: bus -1 (RP06)
acpiprt10 at acpi0: bus -1 (RP07)
acpiprt11 at acpi0: bus -1 (RP08)
acpiec0 at acpi0: not present
acpicpu0 at acpi0: C1(@1 halt!)
acpicpu1 at acpi0: C1(@1 halt!)
acpicpu2 at acpi0: C1(@1 halt!)
acpicpu3 at acpi0: C1(@1 halt!)
acpipwrres0 at acpi0: PG00, resource for PEG0
acpipwrres1 at acpi0: PG01, resource for PEG1
acpipwrres2 at acpi0: PG02, resource for PEG2
acpipwrres3 at acpi0: FN00, resource for FAN0
acpipwrres4 at acpi0: FN01, resource for FAN1
acpipwrres5 at acpi0: FN02, resource for FAN2
acpipwrres6 at acpi0: FN03, resource for FAN3
acpipwrres7 at acpi0: FN04, resource for FAN4
acpitz0 at 

Re: OpenBSD & OpenBGPD router replacement

[Tony Sarendal]

You will likely run out of CPU before bandwidth.
Even on nice hardware I have yet to exceed 1Mpps with OpenBSD.

/T


Den ons 19 dec. 2018 kl 03:12 skrev Max Clark :

> Tom,
>
> The presentation was very interesting and it's given me a lot of food for
> thought for another project. Fortunately for this application I don't need
> to worry about fire walling at the BGP edge, just the router replacement
> itself.
>
> Max
>
>
> On Tue, Dec 18, 2018 at 6:02 PM Tom Smyth 
> wrote:
>
> > Max,
> >
> > another thing to consider, is that with BGP feeds / Advertising
> > you only have some control over which direction traffic enters / leaves
> > the network, you may pefer one transit provider, but another network on
> the
> > internet can prefer your second transit provider, so  you can have
> > traffic that appears out of state...
> >
> > If you need stateful packet filtering  I would suggest stepping that
> > protection
> > back from your edge routers  ... disadvantage is you would need 4
> > devices to do this
> > but this would give you the redundancy you want from a Transit
> perspective
> > and you would have the ability control  flow of traffic between your
> > edge routers
> > and your Stateful Firewalls
> >
> > Hope This Helps
> >
> > Tom Smyth
> >
> >
> >
> >
> > On Wed, 19 Dec 2018 at 01:52, Max Clark  wrote:
> > >
> > > Thanks Arnaud - I understand that it's not a stateful
> protocol/failover.
> > > It's interesting from the standpoint that if I lose a specific box
> acting
> > > as a router I would recover and maintain the route via the affected
> > > carrier. A few minutes of outage for carp and BGP to come up is better
> > than
> > > a prolonged outage until equipment is replaced.
> > >
> > > Max
> > >
> > >
> > > On Tue, Dec 18, 2018 at 4:47 PM Arnaud BRAND  >
> > > wrote:
> > >
> > > > Hi Max,
> > > >
> > > > I would advise against using CARP for BGP peers.
> > > > BGP is a stateful protocol and there's no bgpsyncd, so I don't think
> > > > this
> > > > will work.
> > > >
> > > > I would rather build two servers, and have 2 BGP sessions/fullfeeds,
> > > > each
> > > > on one 10G link in order to provide redundancy.
> > > >
> > > > Best regards
> > > > Arnaud
> > > >
> > > > Le 2018-12-19 00:17, Max Clark a écrit :
> > > > > Hello,
> > > > >
> > > > > I've been presented with an opportunity to greatly simplify
> upstream
> > > > > networking within a datacenter. At this point I'm expecting to
> > condense
> > > > > down to two 10 Gbps full feed IPv4+IPv6 transit links plus a 10
> Gbps
> > > > > link
> > > > > to the peering fabric. Total 95th percentile transit averages in
> the
> > > > > 3-4
> > > > > Gbps range with bursts into the 6-7 Gbps (outside of the rare DDoS
> > then
> > > > > everything just catches on fire until provider mitigation kicks
> in).
> > > > >
> > > > > With the exception of the full tables it's a pretty simple
> > requirement.
> > > > > There's plenty of options to purchase a new TOR device(s) that
> could
> > > > > take
> > > > > the full tables, but I'd just rather not commit the budget for it.
> > Plus
> > > > > this feels like the perfect time to do what I've wanted for a
> while,
> > > > > and
> > > > > deploy an OpenBSD & OpenBGPD edge.
> > > > >
> > > > > I should probably ask first - am I crazy?
> > > > >
> > > > > With that out of the way I could either land the fiber directly
> into
> > > > > NICs
> > > > > on an appropriately sized server, or I was thinking about landing
> the
> > > > > transit links on a 10 Gbps L2 switch and using CARP to provide
> server
> > > > > redundancy on my side (so each transit link would be part of VLAN
> > with
> > > > > two
> > > > > servers connected, primary server would advertise the /30 to the
> > > > > carrier
> > > > > with BGPD, and secondary server could take over with heartbeat
> > > > > failure). I
> > > > > would use two interfaces on the server - one facing the Internet
> and
> > > > > one
> > > > > facing our equipment.
> > > > >
> > > > > Would the access switch in this configuration be a bad idea?
> Should I
> > > > > keep
> > > > > things directly homed on the server?
> > > > >
> > > > > And my last question - are there any specific NICs that I should
> look
> > > > > for
> > > > > and/or avoid when building this?
> > > > >
> > > > > Thanks!
> > > > > Max
> > > >
> >
> >
> >
> > --
> > Kindest regards,
> > Tom Smyth
> >
> > Mobile: +353 87 6193172
> > The information contained in this E-mail is intended only for the
> > confidential use of the named recipient. If the reader of this message
> > is not the intended recipient or the person responsible for
> > delivering it to the recipient, you are hereby notified that you have
> > received this communication in error and that any review,
> > dissemination or copying of this communication is strictly prohibited.
> > If you have received this in error, please notify the sender
> > immediately by telephone at the number above and erase the message
> > You are requested to 

Re: Reduced network performance since installing 6.4

[Tony Sarendal]

Hola,

Unrelated to wifi, I have seen a dramatic drop in forwarding performance in
6.4 and later.
I run some basic performance tests to verify the releases before we deploy
them.
For the same test on the same hardware I have this:

Release, pps
snapshot, 340k
6.4, 340k
6.3, 450k
6.2, 430k
6.1, 420k
6.0, 425k
5.9, 420k
5.8, 450k

In this case the OpenBSD boxes are deployed as firewall clusters, 4x IX in
a LACP trunk, with VLAN interfaces.
6.3 is faster than it looks, in tests like sessions/second it was a lot
faster than 6.2.

/T


OpenBSD 6.4-current (GENERIC.MP) #425: Sun Nov  4 21:32:53 MST 2018
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 34300891136 (32711MB)
avail mem = 33252069376 (31711MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xec170 (34 entries)
bios0: vendor American Megatrends Inc. version "3.0" date 04/24/2015
bios0: Supermicro X10SLD
acpi0 at bios0: rev 2
acpi0: sleep states S0 S4 S5
acpi0: tables DSDT FACP APIC FPDT FIDT SSDT SSDT MCFG PRAD HPET SSDT SSDT
SPMI DMAR EINJ ERST HEST BERT
acpi0: wakeup devices PEGP(S4) PEG0(S4) PEGP(S4) PEG1(S4) PEGP(S4) PEG2(S4)
PXSX(S4) RP01(S4) PXSX(S4) RP02(S4) PXSX(S4) RP03(S4) PXSX(S4) RP04(S4)
PXSX(S4) RP05(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.64 MHz, 06-3c-03
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.00 MHz, 06-3c-03
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.00 MHz, 06-3c-03
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 6 (application processor)
cpu3: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.00 MHz, 06-3c-03
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu3: 256KB 64b/line 8-way L2 cache
cpu3: smt 0, core 3, package 0
ioapic0 at mainbus0: apid 8 pa 0xfec0, version 20, 24 pins
acpimcfg0 at acpi0
acpimcfg0: addr 0xf800, bus 0-63
acpihpet0 at acpi0: 14318179 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 1 (PEG0)
acpiprt2 at acpi0: bus 2 (PEG1)
acpiprt3 at acpi0: bus -1 (PEG2)
acpiprt4 at acpi0: bus 3 (RP01)
acpiprt5 at acpi0: bus -1 (RP02)
acpiprt6 at acpi0: bus -1 (RP03)
acpiprt7 at acpi0: bus -1 (RP04)
acpiprt8 at acpi0: bus -1 (RP05)
acpiprt9 at acpi0: bus -1 (RP06)
acpiprt10 at acpi0: bus -1 (RP07)
acpiprt11 at acpi0: bus -1 (RP08)
acpiec0 at acpi0: not present
acpicpu0 at acpi0: C1(@1 halt!)
acpicpu1 at acpi0: C1(@1 halt!)
acpicpu2 at acpi0: C1(@1 halt!)
acpicpu3 at acpi0: C1(@1 halt!)
acpipwrres0 at acpi0: PG00, resource for PEG0
acpipwrres1 at acpi0: PG01, resource for PEG1
acpipwrres2 at acpi0: PG02, resource for PEG2
acpipwrres3 at acpi0: FN00, resource for FAN0
acpipwrres4 at acpi0: FN01, resource for FAN1
acpipwrres5 at acpi0: FN02, resource for FAN2
acpipwrres6 at acpi0: FN03, resource for FAN3
acpipwrres7 at acpi0: 

Re: Integration between CARP and BGPD ?

[Tony Sarendal]

Or re-write next-hop to the carp address, so carp actually decides the
master firewall.

/T


Den tors 13 sep. 2018 kl 00:20 skrev Tim Jones <
b631093f-779b-4d67-9ffe-5f6d5b1d3...@protonmail.ch>:

>
> On Wednesday, 12 September 2018 20:49, Stuart Henderson <
> s...@spacehopper.org> wrote:
>
> > On 2018-09-11, Tim Jones
> b631093f-779b-4d67-9ffe-5f6d5b1d3...@protonmail.ch wrote:
> >
> > > I've had a quick look through the man pages and am still a bit
> unclear, perhaps I'm just overthinking this ?
> > > Let's say I've got two perimeter "firewalls" running OpenBSD, talking
> BGP to upstream routers.
> > > On the "LAN" side I'm thinking about CARP, which is active/passive,
> and the devices on "LAN" side will have the CARP set as their default
> gateway.
> > > If both BGP talkers advertise the "LAN" to the upstreams (i.e.
> "network 192.0.2.0/24" in bgpd.conf), how does that work in terms of
> reachability from the device that is currently CARP passive ?
> > > The man pages mention two CARP related configuration options for
> bgpd.conf but these don't seem to cater for the application I'm thinking of
> ?  (i.e. "demote" is more related to waiting until BGP is established, and
> "depend on" is related to staying in idle if CARP is passive, which is
> obviously not an attractive idea as I'd obviously like both upstreams BGP
> sessions active ? ).
> >
> > If both are advertising the same prefixes, packets could arrive at
> > either router, so to do this you'll need an IP address on the "carpdev
> > interface" i.e. the interface that carp is running over.
> >
> > PF does TCP sequence number checking, so to avoid problems there you'll
> > also need one of the following
> >
> > -   not use PF
> > -   use PF rules with "keep state (sloppy)"
> > -   use pfsync(4) with the "defer" flag
> >
> > Alternatively maybe you could control advertising the network by not
> > listing it in config, but use "bgpctl network" commands from
> ifstated or
> > similar, that way directing traffic towards the correct machine.
> Either
> > advertise with low localpref when you have carp backup and switch to
> > high localpref when you have master. Or (probably only really useful
> > within your own network) advertise the whole lan all the time, but
> also
> > advertise deaggregates from the machine with carp master.
> >
>
> Thank you Stuart !
>
> Based on your comments I've just spent in a bit of time with ifstated and
> it seems that was the missing link.  Fails over nicely now with both BGP
> instances advertising but changing prefs.
>
>

Re: testing cabling and NIC hardware with one machine

[Tony Sarendal]

Configure the interfaces into separate rdomains.

/T

2017-10-25 21:17 GMT+02:00 Christopher Paul :

> Hi Misc,
>
> I have been tasked with setting up a benchmark platform to test NICs and
> network cables. I'd like to do this on one PC. So I want to send packets of
> different protocols out of one interface and into the other, across/thru
> the NICs and whatever type/lengths of cabling I am using. The problem I am
> having is that if I configure two interfaces on the same server, either on
> the same network or not, the kernel is smart enough to know to not need to
> use the actual wire (ethernet cable) in order to transmit the packet from
> one interface to the other. Which I guess I was aware of, and of course it
> makes a lot of good sense for a normal situation, but in this case, is a
> block on my project.
>
> So I'm wondering is this sort of kernel-fooling I want to do possible with
> OpenBSD? Or for that matter, any OS?
>
> May be I need to set them up as a bridge? If I did that I could set it up
> with forwarding, yeah? Something like that I guess I will try next.
>
> Many thanks for those of you who read this and offer any ideas && Long
> Live OpenBSD,
>
> CP
>
>

ftp.eu.openbsd.org

[Tony Sarendal]

Not looking so good.

tonsar@jump0.swe1$ ftp ftp.eu.openbsd.org
Trying 193.156.26.18...
Connected to ftp.eu.openbsd.org (193.156.26.18).
220 jj-prod-obsdmirror.inet6.se FTP server ready.
Name (ftp.eu.openbsd.org:tonsar): ftp
331 Guest login ok, send your email address as password.
Password:
230 Guest login ok, access restrictions apply.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> dir
227 Entering Passive Mode (192,168,0,13,204,157)
^C

/T

Re: maximum number of interfaces

[Tony Sarendal]

Back in 2007 I tested with 4k VLAN interfaces, it wasn't fast, but it
worked.

/T


2017-04-03 5:46 GMT+02:00 Nick Holland :

> On 04/02/17 22:08, Edgar Pettijohn wrote:
> > Is there a maximum number of network interfaces that can be configured?
> > I looked around in /usr/include to see if I could find it #define(d)
> > somewhere, but couldn't.  I also globbed up my mail server so this is
> > also a test, but I would like to know.
>
> Some time ago (maybe in the 3.5 era), I put five 4-port dc(4) cards in
> one machine, plus a 3com xl(4) chip on the mobo.  Didn't actually DO
> anything with it, but they all counted out just fine.
>
> "lots" :)
>
> Nick.

USB and Intel Bay Trail

[Tony Sarendal]

Hola,

I got a pair of mini-pc's to play with for the summer vacation, small
fanless
thingies with 4xGE and wifi.

http://www.qotom.net/goods-129-QOTOM-Q190G4+4+LAN+Mini+PC.html

When testing with the latest snapshot USB wont play.
Any ideas ?

Regards Tony

# dmesg
OpenBSD 6.0-beta (GENERIC.MP) #2296: Thu Jul 14 20:12:36 MDT 2016
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 8477577216 (8084MB)
avail mem = 8216109056 (7835MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.8 @ 0xebea0 (51 entries)
bios0: vendor American Megatrends Inc. version "5.6.5" date 10/27/2015
acpi0 at bios0: rev 2
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP APIC FPDT FIDT MCFG HPET SSDT SSDT SSDT UEFI
acpi0: wakeup devices XHC1(S4) EHC1(S4) PXSX(S4) PXSX(S4) PXSX(S4) PXSX(S4)
PWRB(S0)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Celeron(R) CPU J1900 @ 1.99GHz, 2000.46 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX
,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,MOVBE,POPCNT,DEADLINE,RDRAND,NXE,
LONG,LAHF,3DNOWP,PERF,ITSC,SMEP,ERMS,SENSOR,ARAT
cpu0: 1MB 64b/line 16-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
cpu0: apic clock running at 83MHz
cpu0: mwait min=64, max=64, C-substates=0.2.0.0.0.0.3.3, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Celeron(R) CPU J1900 @ 1.99GHz, 1999.99 MHz
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX
,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,MOVBE,POPCNT,DEADLINE,RDRAND,NXE,
LONG,LAHF,3DNOWP,PERF,ITSC,SMEP,ERMS,SENSOR,ARAT
cpu1: 1MB 64b/line 16-way L2 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Celeron(R) CPU J1900 @ 1.99GHz, 1999.99 MHz
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX
,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,MOVBE,POPCNT,DEADLINE,RDRAND,NXE,
LONG,LAHF,3DNOWP,PERF,ITSC,SMEP,ERMS,SENSOR,ARAT
cpu2: 1MB 64b/line 16-way L2 cache
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 6 (application processor)
cpu3: Intel(R) Celeron(R) CPU J1900 @ 1.99GHz, 1999.99 MHz
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX
,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,MOVBE,POPCNT,DEADLINE,RDRAND,NXE,
LONG,LAHF,3DNOWP,PERF,ITSC,SMEP,ERMS,SENSOR,ARAT
cpu3: 1MB 64b/line 16-way L2 cache
cpu3: smt 0, core 3, package 0
ioapic0 at mainbus0: apid 1 pa 0xfec0, version 20, 87 pins
acpimcfg0 at acpi0 addr 0xe000, bus 0-255
acpihpet0 at acpi0: 14318179 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 1 (RP01)
acpiprt2 at acpi0: bus 2 (RP02)
acpiprt3 at acpi0: bus 3 (RP03)
acpiprt4 at acpi0: bus 4 (RP04)
acpiec0 at acpi0: not present
acpicpu0 at acpi0: C3(10@1500 mwait.1@0x52), C2(10@500 mwait.1@0x51),
C1(1000@1 mwait.1), PSS
acpicpu1 at acpi0: C3(10@1500 mwait.1@0x52), C2(10@500 mwait.1@0x51),
C1(1000@1 mwait.1), PSS
acpicpu2 at acpi0: C3(10@1500 mwait.1@0x52), C2(10@500 mwait.1@0x51),
C1(1000@1 mwait.1), PSS
acpicpu3 at acpi0: C3(10@1500 mwait.1@0x52), C2(10@500 mwait.1@0x51),
C1(1000@1 mwait.1), PSS
acpipwrres0 at acpi0: PLPE
acpipwrres1 at acpi0: PLPE
acpipwrres2 at acpi0: USBC, resource for EHC1, OTG1
"DMA0F28" at acpi0 not configured
acpibtn0 at acpi0: SLPB
"INT33BD" at acpi0 not configured
acpivideo0 at acpi0: GFX0
acpivout0 at acpivideo0: DD1F
cpu0: Enhanced SpeedStep 2000 MHz: speeds: 1993, 1992, 1909, 1826, 1743,
1660, 1577, 1494, 1411, 1328 MHz
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel Bay Trail Host" rev 0x0e
inteldrm0 at pci0 dev 2 function 0 "Intel Bay Trail Video" rev 0x0e
drm0 at inteldrm0
inteldrm0: msi
inteldrm0: 1920x1200
wsdisplay0 at inteldrm0 mux 1: console (std, vt100 emulation)
wsdisplay0: screen 1-5 added (std, vt100 emulation)
ahci0 at pci0 dev 19 function 0 "Intel Bay Trail AHCI" rev 0x0e: msi, AHCI
1.3
ahci0: port 0: 3.0Gb/s
scsibus1 at ahci0: 32 targets
sd0 at scsibus1 targ 0 lun 0:  SCSI3 0/direct
fixed naa.5002538043584d30
sd0: 61057MB, 512 bytes/sector, 125045424 sectors, thin
"Intel Bay Trail TXE" rev 0x0e at pci0 dev 26 function 0 not configured
azalia0 at pci0 dev 27 function 0 "Intel Bay Trail HD Audio" rev 0x0e: msi
azalia0: no supported codecs
ppb0 at pci0 dev 28 function 0 "Intel Bay Trail I2C" rev 0x0e: msi
pci1 at ppb0 bus 1
em0 at pci1 dev 0 function 0 "Intel 82583V" rev 0x00: msi, address
00:ba:ab:10:24:e0
ppb1 at 

lots of states (5.8)

[Tony Sarendal]

Hola amigos,

I'm doing some testing in the lab at the moment and just though I'd share.

pf0.swe69# pfctl -si | grep current
  current entries 50239413
pf0.swe69# vmstat -m | tail -n 1
In use 22035659K, total allocated 5678936K; utilization 388.0%
pf0.swe69#

4 tcpbench sessions through it (450kpps under normal running) :

pf0.swe69# netstat -I trunk0 -w1
trunk in  trunk out  total in  total out
 packets  errs  packets  errs colls   packets  errs  packets  errs colls
28547079349 0 32372963420  2822 0  114224543321 4 101542519853
 6675 0
  407927 0   408373 0 0   1631255 0  1225979 0 0
  413105 0   414141 0 0   1652684 0  1242994 0 0
  404324 0   404859 0 0   1617350 0  1215559 0 0
  408613 0   409500 0 0   1634610 0  1229346 0 0
  406545 0   407357 0 0   1626177 0  1222868 0 0
  412529 0   413248 0 0   1649941 0  1240605 0 0
  406656 0   407405 0 0   1626810 0  1222997 0 0
  411297 0   412122 0 0   1645393 0  1237101 0 0

httperf can maintain a session rate of 650 sessions per second (12k+ under
normal running):

cloud8.swe69$ route -T2 exec httperf --server 10.96.2.24 --uri /1k.bin
--num-conns 25000 --rate 650
httperf --client=0/1 --server=10.96.2.24 --port=80 --uri=/1k.bin --rate=650
--send-buffer=4096 --recv-buffer=16384 --num-conns=25000 --num-calls=1
Maximum connect burst length: 2

Total: connections 25000 requests 25000 replies 25000 test-duration 44.214 s

Connection rate: 565.4 conn/s (1.8 ms/conn, <=381 concurrent connections)
Connection time [ms]: min 190.9 avg 261.3 max 6408.1 median 198.5 stddev
334.3
Connection time [ms]: connect 43.0
Connection length [replies/conn]: 1.000

Request rate: 565.4 req/s (1.8 ms/req)
Request size [B]: 69.0

Reply rate [replies/s]: min 473.8 avg 624.6 max 672.6 stddev 64.4 (8
samples)
Reply time [ms]: response 1.2 transfer 217.1
Reply size [B]: header 211.0 content 1024.0 footer 0.0 (total 1235.0)
Reply status: 1xx=0 2xx=25000 3xx=0 4xx=0 5xx=0

CPU time [s]: user 0.63 system 43.59 (user 1.4% system 98.6% total 100.0%)
Net I/O: 720.0 KB/s (5.9*10^6 bps)

Errors: total 0 client-timo 0 socket-timo 0 connrefused 0 connreset 0
Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0



50M+ states and still standing, pretty good I think.

/T



OpenBSD 5.8-stable (GENERIC.MP) #103: Mon May  9 12:15:30 CEST 2016
root@ob2.swe69:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 34300891136 (32711MB)
avail mem = 33257451520 (31716MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xec170 (34 entries)
bios0: vendor American Megatrends Inc. version "3.0" date 04/24/2015
bios0: Supermicro X10SLD
acpi0 at bios0: rev 2
acpi0: sleep states S0 S4 S5
acpi0: tables DSDT FACP APIC FPDT FIDT SSDT SSDT SSDT SSDT MCFG PRAD HPET
SSDT SSDT SPMI DMAR EINJ ERST HEST BERT
acpi0: wakeup devices PEGP(S4) PEG0(S4) PEGP(S4) PEG1(S4) PEGP(S4) PEG2(S4)
PXSX(S4) RP01(S4) PXSX(S4) RP02(S4) PXSX(S4) RP03(S4) PXSX(S4) RP04(S4)
PXSX(S4) RP05(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.61 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX
,SMX,EST,TM2,SSSE3,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT
,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,ABM,PERF,ITSC,FSGSB
ASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,SENSOR,ARAT
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.00 MHz
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX
,SMX,EST,TM2,SSSE3,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT
,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,ABM,PERF,ITSC,FSGSB
ASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,SENSOR,ARAT
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.00 MHz
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX
,SMX,EST,TM2,SSSE3,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT
,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,ABM,PERF,ITSC,FSGSB

Re: bgpd network connected

[Tony Sarendal]

2016-03-08 15:38 GMT+01:00 Matt Schwartz <matt.schwart...@gmail.com>:

> I did not even know it was broken?
>
> On Mar 8, 2016 1:26 AM, "Tony Sarendal" wrote:
> >
> > Is there any chance of getting "network inet connected" fixed to 5.9 ?
> >
> > Regards Tony
>
>

Adding a new vlan interface:

beer# cat /etc/bgpd.conf
AS 65001
network inet connected
beer# bgpctl show rib
flags: * = Valid, > = Selected, I = via IBGP, A = Announced, S = Stale
origin: i = IGP, e = EGP, ? = Incomplete

flags destination  gateway  lpref   med aspath origin
AI*>  172.29.1.0/240.0.0.0100 0 i
beer# ifconfig vlan69 create
beer# ifconfig vlan69 1.1.1.1/30 vlandev em0 vlan 69 up
beer# bgpctl show rib
flags: * = Valid, > = Selected, I = via IBGP, A = Announced, S = Stale
origin: i = IGP, e = EGP, ? = Incomplete

flags destination  gateway  lpref   med aspath origin
AI*>  172.29.1.0/240.0.0.0100 0 i
beer# /etc/rc.d/bgpd restart
beer# bgpctl show rib
flags: * = Valid, > = Selected, I = via IBGP, A = Announced, S = Stale
origin: i = IGP, e = EGP, ? = Incomplete

flags destination  gateway  lpref   med aspath origin
AI*>  1.1.1.0/30   0.0.0.0100 0 i
AI*>  172.29.1.0/240.0.0.0100 0 i
beer#


Regards Tony

bgpd network connected

[Tony Sarendal]

Is there any chance of getting "network inet connected" fixed to 5.9 ?

Regards Tony

Re: openbgpd puts wrong nexthop in FIB

[Tony Sarendal]

2016-01-21 11:16 GMT+01:00 Stuart Henderson <s...@spacehopper.org>:

> On 2016-01-20, Tony Sarendal <t...@polarcap.org> wrote:
> > network inet connected is broken in 5.6, 5.8 and -current.
> > Restarting bgpd is required when making interface changes.
>
> Ah, so it was fixed in 5.7 and broken again? Now the previous mail
> (http://permalink.gmane.org/gmane.os.openbsd.misc/227597) makes more
> sense.
>
>
That sums is up well. Serious a bug in a piece of routing software.

/T

Re: openbgpd puts wrong nexthop in FIB

[Tony Sarendal]

network inet connected is broken in 5.6, 5.8 and -current.
Restarting bgpd is required when making interface changes.

/T

2016-01-20 20:36 GMT+01:00 Denis Fondras :

> Hello,
>
> I'm using -current as a BGP router and "sometimes" it won't put the right
> nexthop in FIB. The only thing I played with is the interface that support
> IP
> 185.1.2.12 (ifconfig up/down/delete ip /add ip). Anybody can reproduce ?
>
> # bgpctl sh rib 185.22.131.1
> flags: * = Valid, > = Selected, I = via IBGP, A = Announced, S = Stale
> origin: i = IGP, e = EGP, ? = Incomplete
>
> flags destination  gateway  lpref   med aspath origin
> *>185.22.131.0/24  185.1.2.10 100 0 199881 i
>
> # bgpctl sh fib 185.22.131.1
> flags: * = valid, B = BGP, C = Connected, S = Static, D = Dynamic
>N = BGP Nexthop reachable via this route R = redistributed
>r = reject route, b = blackhole route
>
> flags prio destination  gateway
> *B  48 185.22.131.0/24  185.1.2.12
>
> Denis

5.8 bgpd, network connected behaves like 5.6

[Tony Sarendal]

"network inet connected" does not pick up new vlan interfaces, same problem
as 5.6.

bmr0.esp1# ifconfig vlan69 create
bmr0.esp1# ifconfig vlan69 vlandev trunk0 vlan 69 up
bmr0.esp1# ifconfig vlan69 1.1.1.1/30
bmr0.esp1# bgpctl show rib 1.1.1.1
flags: * = Valid, > = Selected, I = via IBGP, A = Announced, S = Stale
origin: i = IGP, e = EGP, ? = Incomplete

flags destination  gateway  lpref   med aspath origin
bmr0.esp1# bgpctl show int
Interface  Nexthop state  Flags  Link state
vlan69 ok UP Ethernet, active
vlan105ok UP Ethernet, active
vlan104ok UP Ethernet, active
gre0   ok UP unknown
pflog0 ok UP unknown
vlan103ok UP Ethernet, active
trunk0 ok UP Ethernet, active
lo1ok UP unknown
lo0ok UP unknown
enc0   invalid   active
ix1ok UP active, 10 GBit/s
ix0ok UP active, 10 GBit/s
bmr0.esp1#

As with 5.6, restarting bgpd fixes it:
bmr0.esp1# /etc/rc.d/bgpd restart
bgpd(ok)
bgpd(ok)
bmr0.esp1# bgpctl show rib 1.1.1.1
flags: * = Valid, > = Selected, I = via IBGP, A = Announced, S = Stale
origin: i = IGP, e = EGP, ? = Incomplete

flags destination  gateway  lpref   med aspath origin
AI*>  1.1.1.0/30   0.0.0.0100 0 i
bmr0.esp1#


Any chance of getting a fix into -stable 5.8 ?

/T

OpenBSD 5.8-stable (GENERIC.MP) #16: Tue Dec  8 10:44:35 CET 2015
r...@obc0.rad.unibet.com:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 34300891136 (32711MB)
avail mem = 33257451520 (31716MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xec170 (34 entries)
bios0: vendor American Megatrends Inc. version "3.0" date 04/24/2015
bios0: Supermicro X10SLD
acpi0 at bios0: rev 2
acpi0: sleep states S0 S4 S5
acpi0: tables DSDT FACP APIC FPDT FIDT SSDT SSDT SSDT SSDT SSDT MCFG PRAD
HPET SSDT SSDT SPMI DMAR EINJ ERST HEST BERT
acpi0: wakeup devices PEGP(S4) PEG0(S4) PEGP(S4) PEG1(S4) PEGP(S4) PEG2(S4)
PXSX(S4) RP01(S4) PXSX(S4) RP02(S4) PXSX(S4) RP03(S4) PXSX(S4) RP04(S4)
PXSX(S4) RP05(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.42 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX
,SMX,EST,TM2,SSSE3,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT
,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,ABM,PERF,ITSC,FSGSB
ASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,SENSOR,ARAT
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 100MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.01 MHz
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX
,SMX,EST,TM2,SSSE3,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT
,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,ABM,PERF,ITSC,FSGSB
ASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,SENSOR,ARAT
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.01 MHz
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX
,SMX,EST,TM2,SSSE3,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT
,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,ABM,PERF,ITSC,FSGSB
ASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,SENSOR,ARAT
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 6 (application processor)
cpu3: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.01 MHz
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX
,SMX,EST,TM2,SSSE3,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT
,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,LONG,LAHF,ABM,PERF,ITSC,FSGSB
ASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,SENSOR,ARAT
cpu3: 256KB 64b/line 8-way L2 cache
cpu3: smt 0, core 3, package 0
cpu4 at mainbus0: apid 1 (application processor)
cpu4: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.01 MHz
cpu4:

Re: 5.8 bgpd, network connected behaves like 5.6

[Tony Sarendal]

2015-12-17 10:29 GMT+01:00 Peter Hessler :

> 1) does "bgpctl reload" detect it?
>
> 2) does -current work as you expect?
>
>
>
1. bgpctl reload does not make any difference.

2. A quick test on my -current workstation (not the same hardware, no
trunk) also fails to work.
-current from the 14th.

/T

delorean# cat /etc/bgpd.conf
AS 65001
network inet connected
network inet static

delorean# bgpctl show rib
flags: * = Valid, > = Selected, I = via IBGP, A = Announced, S = Stale
origin: i = IGP, e = EGP, ? = Incomplete

flags destination  gateway  lpref   med aspath origin
AI*>  10.96.5.0/24 0.0.0.0100 0
delorean# bgpctl show int
Interface  Nexthop state  Flags  Link state
pflog0 ok UP unknown
lo0ok UP unknown
enc0   invalid   active
em0ok UP Ethernet, active, 1000 MBit/s
delorean#
delorean#
delorean# ifconfig vlan69 create
delorean# ifconfig vlan69 vlandev em0 vlan 69 up
delorean# ifconfig vlan69 1.1.1.1/30
delorean#
delorean# bgpctl show rib
flags: * = Valid, > = Selected, I = via IBGP, A = Announced, S = Stale
origin: i = IGP, e = EGP, ? = Incomplete

flags destination  gateway  lpref   med aspath origin
AI*>  10.96.5.0/24 0.0.0.0100 0 i
delorean# bgpctl show int
Interface  Nexthop state  Flags  Link state
vlan69 ok UP Ethernet, active, 1000 MBit/s
pflog0 ok UP unknown
lo0ok UP unknown
enc0   invalid   active
em0ok UP Ethernet, active, 1000 MBit/s
delorean#
delorean#
delorean#
delorean# bgpctl reload
reload request sent.
request processed
delorean# bgpctl show rib
flags: * = Valid, > = Selected, I = via IBGP, A = Announced, S = Stale
origin: i = IGP, e = EGP, ? = Incomplete

flags destination  gateway  lpref   med aspath origin
AI*>  10.96.5.0/24 0.0.0.0100 0 i
delorean# bgpctl show int
Interface  Nexthop state  Flags  Link state
vlan69 ok UP Ethernet, active, 1000 MBit/s
pflog0 ok UP unknown
lo0ok UP unknown
enc0   invalid   active
em0ok UP Ethernet, active, 1000 MBit/s
delorean#
delorean#
delorean#
delorean# /etc/rc.d/bgpd restart
bgpd(ok)
bgpd(ok)
delorean# bgpctl show rib
flags: * = Valid, > = Selected, I = via IBGP, A = Announced, S = Stale
origin: i = IGP, e = EGP, ? = Incomplete

flags destination  gateway  lpref   med aspath origin
AI*>  1.1.1.0/30   0.0.0.0100 0 i
AI*>  10.96.5.0/24 0.0.0.0100 0 i
delorean# bgpctl show int
Interface  Nexthop state  Flags  Link state
vlan69 ok UP Ethernet, active, 1000 MBit/s
pflog0 ok UP unknown
lo0ok UP unknown
enc0   invalid   active
em0ok UP Ethernet, active, 1000 MBit/s
delorean#

OpenBSD 5.8-current (GENERIC.MP) #13: Mon Dec 14 12:11:14 CET 2015
r...@delorean.rad.unibet.com:/usr/src/sys/arch/amd64/compile/GENERIC.MP
RTC BIOS diagnostic error d0
real mem = 4161183744 (3968MB)
avail mem = 4030943232 (3844MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xe87ec (85 entries)
bios0: vendor Hewlett-Packard version "J01 v02.15" date 11/10/2011
bios0: Hewlett-Packard HP Compaq 8200 Elite SFF PC
acpi0 at bios0: rev 2
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP APIC SSDT MCFG HPET ASF! SSDT SLIC TCPA DMAR
acpi0: wakeup devices PS2K(S3) PS2M(S3) BR20(S4) EUSB(S3) USBE(S3) PEX0(S4)
PEX1(S4) PEX2(S4) PEX3(S4) PEX4(S4) PEX5(S4) PEX6(S4) PEX7(S4) GBE_(S4)
P0P1(S4) P0P2(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM) i5-2500 CPU @ 3.30GHz, 3293.46 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX
,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,A
ES,XSAVE,AVX,NXE,LONG,LAHF,PERF,ITSC,SENSOR,ARAT
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.1, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Core(TM) i5-2500 CPU @ 3.30GHz, 3292.50 MHz
cpu1:

Re: Donation request for Network SMP development

[Tony Sarendal]

How is this going ?

/T


On Fri, Mar 20, 2015 at 8:57 PM, Martin Pieuchot m...@openbsd.org wrote:

 If you've been following my contributions to OpenBSD's kernel, you
 already know that in the past years I've been working on the Network
 Stack [1] to make it more SMP friendly [2].

 All the network hackers present at s2k15 agreed to volunteer me to work
 on the next step: properly integrate the pseudo-drivers (carp(4),
 vlan(4), trunk(4)...) in order to take ether_input() out of the kernel
 lock.

 But since I no longer have the support of a company, I don't have the
 correct toys to do this task.  That's why I'm looking for the following
 hardware, to build a crazy test  development CARP setup:

   -  A small managed switch (8+ ports) preferably with a CLI interface
  like HP Procurves 25xx.

   - Two small fanless MP amd64 machines with 3+ NIC and a serial console
 like PC Engines APU or Lanner LEC.

 I'm based in Europe, please contact me if you can help out.

 Thanks,
 Martin


 [1] http://www.openbsd.org/papers/tamingdragons.pdf
 [2] http://undeadly.org/cgi?action=articlesid=20150218085759

bgpd.conf macros on 5.5 and up

[Tony Sarendal]

From 5.5 and up it looks like bgpd macros are broken.

ton...@obc2.rad$ cat bgpd.conf
good={ 192.168.0.0/16, 172.16.0.0/12, 10.0.0.0/8 }
AS 65001
deny from any prefix { $good }
ton...@obc2.rad$

On 5.4:
ton...@obc2.rad$ bgpd -f bgpd.conf
-n
configuration OK
ton...@obc2.rad$

On 5.5:
ton...@obc0.rad$ bgpd -f bgpd.conf -nv
good = { 192.168.0.0/16, 172.16.0.0/12, 10.0.0.0/8 }
bgpd.conf:3: syntax error
ton...@obc0.rad$

On 5.6 snapshot:
tonsar@obc1$ uname -mrsv
OpenBSD 5.6 GENERIC.MP#701 amd64
tonsar@obc1$ bgpd -f bgpd.conf -nv
good = { 192.168.0.0/16, 172.16.0.0/12, 10.0.0.0/8 }
bgpd.conf:3: syntax error
tonsar@obc1$


Also, the example from bgpd.conf man page fails on 5.4-5.6.
I haven't tested on 5.3 and lower.

On 5.6 snapshot:
tonsar@obc1$ uname -mrsv
OpenBSD 5.6 GENERIC.MP#701 amd64
tonsar@obc1$ cat bgpd.conf-2
good={ 192.168.0.0/16, 172.16.0.0/12, 10.0.0.0/8 }
bad={ 224.0.0.0/4 prefixlen = 4, 240.0.0.0/4 prefixlen = 4 }
ugly={ 127.0.0.1/8, 169.254.0.0/16 }
# global configuration
AS 65001
deny from any prefix { $good $bad $ugly }
tonsar@obc1$ bgpd -f bgpd.conf-2 -nv
good = { 192.168.0.0/16, 172.16.0.0/12, 10.0.0.0/8 }
bad = { 224.0.0.0/4 prefixlen = 4, 240.0.0.0/4 prefixlen = 4 }
ugly = { 127.0.0.1/8, 169.254.0.0/16 }
bgpd.conf-2:6: syntax error
tonsar@obc1$


Regards Tony

4k graphics and openbsd

[Tony Sarendal]

Good afternoon,

Friday question:
Does anyone have recommendation on graphics hardware to use for 4k screens
and OpenBSD ?

I'm thinking about improving my workstation. I run lots of terminal
windows, a web browser,
and the default window manager. As I like eye candy I may even do xsetroot
-solid black.

What I want is a stable work environment where I can reboot my workstation
every
6 months or so. This with a 4k screen. Doable ?

Cheers

/Tony

Re: 4k graphics and openbsd

[Tony Sarendal]

On Fri, Sep 19, 2014 at 6:07 PM, Jonathan Gray j...@jsg.id.au wrote:

 On Fri, Sep 19, 2014 at 02:22:49PM +0200, Tony Sarendal wrote:
  Good afternoon,
 
  Friday question:
  Does anyone have recommendation on graphics hardware to use for 4k
 screens
  and OpenBSD ?
 
  I'm thinking about improving my workstation. I run lots of terminal
  windows, a web browser,
  and the default window manager. As I like eye candy I may even do
 xsetroot
  -solid black.
 
  What I want is a stable work environment where I can reboot my
 workstation
  every
  6 months or so. This with a 4k screen. Doable ?

 The short version is it won't work yet, the longer version
 is you should carefully check what the graphics hardware, monitor
 and software support.

 For Intel the HDMI 4k at 30 Hz modes are only supported on Haswell,
 ie i[357]-4xxx.  On the AMD side HDMI 4k at 30 Hz modes are only
 supported on Southern Islands parts according to
 http://xorg.freedesktop.org/wiki/RadeonFeature/ which are
 Radeon HD = 77xx.  The Southern Islands parts are best avoided as
 xf86-video-ati only supports 2D acceleration via Glamor-EGL which on
 Southern Islands radeons requires a Mesa driver that has a hard
 requirement on LLVM, working EGL drm platform, and other things.
 And the kernel only knows about the initial round of Southern Islands
 parts, not all of those that were later released including
 the newer integrated graphics with Kaveri.  It seems it should
 be possible to use 4k modes with displayport on Northern Islands/HD6xxx
 if the screen was presented as two smaller screens via MST as
 Northern Islands radeons support displayport 1.2 and MST.
 This isn't an option for older Intel parts as ivybridge for
 example only supports displayport 1.1 with no MST.

 The HDMI 1.4 4k modes make use of a CEA vendor extension
 in the blob of data from the display that describes the
 modes (EDID).  Support for that was added in drm 3.12.

 Many of the 4k displays present themselves as multiple
 displayport displays/streams, support for that was added
 in drm 3.17.

 OpenBSD has roughly drm 3.8.13.28 at the moment.

 hdmi 1.4, drm 3.12
 3840x2160@30Hz
 3840x2160@25Hz
 3840x2160@24Hz

 hdmi 2.0
 4k @ 60Hz, drm ?

 displayport 1.2 4k 30Hz single stream transport (SST), drm 3.12?

 displayport 1.2 4k 60Hz multi stream transport (MST), drm 3.17

 displayport 1.? 4k 60Hz single stream transport (4K60 SST), drm ?

 displayport 1.? dell 5k 5120x2880@?, MST? drm ?



Thanks for taking the time, Jonathan.

Regards Tony

Re: packets logged by pf without log rule

[Tony Sarendal]

On Tue, Sep 16, 2014 at 12:20 AM, Alexander Salmin alexan...@salmin.biz
wrote:

 Did you see it in previous versions?
 I would compare the same ruleset with a fresh 5.5 and see if you
 experience the same and in that case continue compare the relevant
 sourcecode.


The behaviour is the same as far back as 5.4 at least.

I have another one. With the pass quick all rule-set. of I send:
09:34:28.490074 00:25:90:c1:f1:8c 01:00:5e:40:68:01 0800 1514:
10.69.48.14.5404  239.192.104.1.5405: udp 1473 (frag 49575:1480@0+) [ttl 1]
twice within 60s (frag timer ?)

I get:
Sep 16 09:34:28.490095 rule def/(match) pass in on em0: 10.69.48.14.5404 
239.192.104.1.5405: udp 1473 (frag 49575:1480@0+) [ttl 1]

I see this a lot in our production and test environment, but there it is
triggered without the duplicate packet.

Example from live firewall. Traffic:
pf0.swe1# tcpdump -n -i vlan57 host 10.69.48.14 and not tcp
tcpdump: listening on vlan57, link-type EN10MB
tcpdump: WARNING: compensating for unaligned libpcap packets
09:51:56.710780 10.69.48.14.5404  239.192.104.1.5405: udp 75 (DF) [ttl 1]
09:51:56.711161 10.69.48.14.5404  239.192.104.1.5405: udp 1473 (frag
27013:1480@0+) [ttl 1]
09:51:56.711163 10.69.48.14  239.192.104.1: (frag 27013:1@1480) [ttl 1]
09:51:56.711164 10.69.48.14.5404  239.192.104.1.5405: udp 1473 (frag
27014:1480@0+) [ttl 1]
09:51:56.711166 10.69.48.14  239.192.104.1: (frag 27014:1@1480) [ttl 1]
09:51:56.711167 10.69.48.14.5404  239.192.104.1.5405: udp 1473 (frag
27015:1480@0+) [ttl 1]
09:51:56.711168 10.69.48.14  239.192.104.1: (frag 27015:1@1480) [ttl 1]
09:51:56.711169 10.69.48.14.5404  239.192.104.1.5405: udp 1473 (frag
27016:1480@0+) [ttl 1]
09:51:56.711171 10.69.48.14  239.192.104.1: (frag 27016:1@1480) [ttl 1]
09:51:56.711172 10.69.48.14.5404  239.192.104.1.5405: udp 1473 (frag
27017:1480@0+) [ttl 1]
09:51:56.711173 10.69.48.14  239.192.104.1: (frag 27017:1@1480) [ttl 1]
09:51:56.711175 10.69.48.14.5404  239.192.104.1.5405: udp 617 (DF) [ttl 1]
09:51:56.713383 10.69.48.14.5404  239.192.104.1.5405: udp 753 (DF) [ttl 1]
09:51:56.724606 10.69.48.14.5404  239.192.104.1.5405: udp 1473 (frag
27018:1480@0+) [ttl 1]
09:51:56.724608 10.69.48.14  239.192.104.1: (frag 27018:1@1480) [ttl 1]
09:51:56.724609 10.69.48.14.5404  239.192.104.1.5405: udp 707 (DF) [ttl 1]
09:51:56.724986 10.69.48.14.5404  239.192.104.1.5405: udp 1412 (DF) [ttl 1]
09:51:56.730168 10.69.48.14.5404  239.192.104.1.5405: udp 650 (DF) [ttl 1]
^C


Log:
pf0.swe1# tcpdump -n -e -ttt -i pflog0 host 10.69.48.14
tcpdump: WARNING: snaplen raised from 116 to 160
tcpdump: listening on pflog0, link-type PFLOG
Sep 16 09:51:56.711185 rule def/(match) pass in on vlan57: 10.69.48.14.5404
 239.192.104.1.5405: udp 1473 (frag 27013:1480@0+) [ttl 1]
tcpdump: WARNING: compensating for unaligned libpcap packets
Sep 16 09:51:56.711190 rule def/(match) pass in on vlan57: 10.69.48.14.5404
 239.192.104.1.5405: udp 1473 (frag 27014:1480@0+) [ttl 1]
Sep 16 09:51:56.711194 rule def/(match) pass in on vlan57: 10.69.48.14.5404
 239.192.104.1.5405: udp 1473 (frag 27015:1480@0+) [ttl 1]
Sep 16 09:51:56.711198 rule def/(match) pass in on vlan57: 10.69.48.14.5404
 239.192.104.1.5405: udp 1473 (frag 27016:1480@0+) [ttl 1]
Sep 16 09:51:56.711202 rule def/(match) pass in on vlan57: 10.69.48.14.5404
 239.192.104.1.5405: udp 1473 (frag 27017:1480@0+) [ttl 1]
Sep 16 09:51:56.724622 rule def/(match) pass in on vlan57: 10.69.48.14.5404
 239.192.104.1.5405: udp 1473 (frag 27018:1480@0+) [ttl 1]
^C
20 packets received by filter
0 packets dropped by kernel
pf0.swe1#

There is no rule that should log this in the live firewalls.
Happens on 5.4 and 5.5, if memory serves me right I saw it on 5.3's also.

Assistance with understanding this would be appreciated.
I will use free time slots to look at the code, but due to limited
knowledge and skills it is quite time consuming.

Regards Tony

packets logged by pf without log rule

[Tony Sarendal]

I'm currently looking into some logging strangeness in we are seeing.
Does anyone know why this is logged ?

obc3.rad# cat /etc/pf.conf
pass quick all
obc3.rad# pfctl -sr
pass quick all flags S/SA
obc3.rad# tcpdump -n -e -ttt -i pflog0
tcpdump: WARNING: snaplen raised from 116 to 160
tcpdump: listening on pflog0, link-type PFLOG
Sep 15 16:07:31.276913 rule 0/(match) pass in on em0: 10.69.48.14 
239.192.104.1: igmp nreport 239.192.104.1 (DF) [tos 0xc0] [ttl 1]
Sep 15 16:07:31.278020 rule 0/(match) pass in on em0: 10.69.48.14 
239.192.104.1: igmp nreport 239.192.104.1 (DF) [tos 0xc0] [ttl 1]


obc3.rad# tcpdump -n -i em0 igmp
tcpdump: listening on em0, link-type EN10MB
tcpdump: WARNING: compensating for unaligned libpcap packets
16:07:31.276905 10.69.48.14  239.192.104.1: igmp nreport 239.192.104.1
(DF) [tos 0xc0] [ttl 1]
16:07:31.278014 10.69.48.14  239.192.104.1: igmp nreport 239.192.104.1
(DF) [tos 0xc0] [ttl 1]


Regards Tony


OpenBSD 5.6-current (GENERIC.MP) #0: Wed Sep 10 13:39:02 CEST 2014
r...@obc3.rad.unibet.com:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 8545173504 (8149MB)
avail mem = 8308969472 (7924MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xeb4c0 (54 entries)
bios0: vendor American Megatrends Inc. version 2.0a date 06/08/2012
bios0: Supermicro X9SCD
acpi0 at bios0: rev 2
acpi0: sleep states S0 S1 S4 S5
acpi0: tables DSDT FACP APIC FPDT MCFG HPET SSDT PRAD SPMI SSDT SPCR EINJ
ERST HEST BERT BGRT
acpi0: wakeup devices PS2K(S4) PS2M(S4) UAR1(S4) P0P1(S4) USB1(S4) USB2(S4)
USB3(S4) USB4(S4) USB5(S4) USB6(S4) USB7(S4) PXSX(S4) RP01(S4) PXSX(S4)
RP02(S4) PXSX(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(R) CPU E3-1270 V2 @ 3.50GHz, 3500.49 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX
,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,A
ES,XSAVE,AVX,F16C,RDRAND,NXE,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 100MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.1.0, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Xeon(R) CPU E3-1270 V2 @ 3.50GHz, 3500.02 MHz
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX
,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,A
ES,XSAVE,AVX,F16C,RDRAND,NXE,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Xeon(R) CPU E3-1270 V2 @ 3.50GHz, 3500.02 MHz
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX
,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,A
ES,XSAVE,AVX,F16C,RDRAND,NXE,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 6 (application processor)
cpu3: Intel(R) Xeon(R) CPU E3-1270 V2 @ 3.50GHz, 3500.02 MHz
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX
,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,A
ES,XSAVE,AVX,F16C,RDRAND,NXE,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS
cpu3: 256KB 64b/line 8-way L2 cache
cpu3: smt 0, core 3, package 0
cpu4 at mainbus0: apid 1 (application processor)
cpu4: Intel(R) Xeon(R) CPU E3-1270 V2 @ 3.50GHz, 3500.02 MHz
cpu4:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX
,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,A
ES,XSAVE,AVX,F16C,RDRAND,NXE,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS
cpu4: 256KB 64b/line 8-way L2 cache
cpu4: smt 1, core 0, package 0
cpu5 at mainbus0: apid 3 (application processor)
cpu5: Intel(R) Xeon(R) CPU E3-1270 V2 @ 3.50GHz, 3500.02 MHz
cpu5:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS
H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX
,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,A
ES,XSAVE,AVX,F16C,RDRAND,NXE,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS
cpu5: 256KB 64b/line 8-way L2 cache
cpu5: smt 1, core 1, package 0
cpu6 at mainbus0: apid 5 (application processor)
cpu6: Intel(R) Xeon(R) CPU E3-1270 V2 @ 3.50GHz, 3500.02 MHz
cpu6:

Re: bgpctl show advertisements?

[Tony Sarendal]

bgpctl show rib nei neighbor out


On Mon, Sep 15, 2014 at 3:55 AM, Adam Thompson athom...@athompso.net
wrote:

 Is there any functionality in bgpctl(8) that will show me precisely what
 I'm advertising to a neighbor?
 If not, is there any easier way - assuming I don't have access to my
 neighbor's router, and they don't run a looking-glass - to find that out,
 short of packet sniffing?

 --
 -Adam Thompson
  athom...@athompso.net

Re: pfsync and trunk

[Tony Sarendal]

On Sat, Sep 13, 2014 at 10:17 AM, Henning Brauer hb-open...@ml.bsws.de
wrote:

 * Tony Sarendal t...@polarcap.org [2014-09-03 06:48]:
  The initial request disappearing and the firewalls staying demoted
  forever are independent issues.

 sure about that? the demotion counter for the interface group pfsyncX
 is part of (usually carp) is kept raised until the bulk transfer
 finishes.


Looks like separate issues.

I have done more testing since, using 5.5. In all cases the demote
counter restores after the bulk transfer completes, or after pfsync
gives up after 12 retries. I have a few 5.4 where I can't explain the
33 demote counter, but I can't replicate it when testing.

I could replicate the problem with the initial request disappearing when
using trunk.
The sleep solves it, I can live with that.

On our heavier firewalls bulk transfer never completes, but carp restores
after
the 12 retries, close to 29h after reboot.

/T

Re: pfsync and trunk

[Tony Sarendal]

As Chuck pointed out this has nothing to do with pfsense or freebsd.

While I dig deeper I'm running with the following config to get around the
problem:
pf1.swe1# cat /etc/hostname.pfsync0
! sleep 10
! ifconfig $1 syncdev vlan44 syncpeer 10.240.252.77 up

pf1.swe1#

I see the request for the bulk transfer now, and the bulk transfer starting.
Although bulk transfer performance looks like a problem, but that is for
another thread.

/T



On Sat, Aug 30, 2014 at 9:31 PM, System Administrator ad...@bitwise.net
wrote:

 And what does OP's message have to do with pfSense ??? (especially
 since he's clearly indicating currently supported OpenBSD versions 5.4
 and 5.5 near the bottom...)

 On 30 Aug 2014 at 14:22, Chuck Burns wrote:

  On Saturday, August 30, 2014 8:27:24 AM Tony Sarendal wrote:
   Good morning,
  
   I'm having issues with pfsync on trunk interfaces, although I suspect
   it to
  snip
   Running on pfsync on trunk(4) that initial request never shows up, and
   the bulk update never starts/finishes. I would like to run pfsync on
   trunk(4) lacp link, but as it looks now I have firewalls with carp
   demote counter 33 forever.
  snip
 
  pfSense is FreeBSD-based. not OpenBSD-based...
 
  different versions of pf between OpenBSD and FreeBSD
 
  --
  Chuck Burns
  Audemus Jura Nostra Defendere

Re: pfsync and trunk

[Tony Sarendal]

Final email in this thread, for correctness.

The initial request disappearing and the firewalls staying demoted
forever are independent issues.
A new request for bulk transfer is sent after 2h+. Due to bulk transfer
performance the transfers
never finish. I see on average 3kpps of pfsync on this cluster, looking at
pfsync this is what I find:

12:02:45.778145 10.240.252.78  10.240.252.77: PFSYNCv6 len 36
act UPD ST REQ count 1
id:  creatorid: 

12:02:45.778153 10.240.252.77  10.240.252.78: PFSYNCv6 len 1412
act BULK UPD STAT count 1
creatorid: b33d7f45 age: 00:00:00 status: start

14:16:09.689102 10.240.252.78  10.240.252.77: PFSYNCv6 len 1264
act UPD ST REQ count 1
id:  creatorid: 

14:16:09.689114 10.240.252.77  10.240.252.78: PFSYNCv6 len 124
act BULK UPD STAT count 1
creatorid: b33d7f45 age: 00:00:00 status: start

16:29:33.604110 10.240.252.78  10.240.252.77: PFSYNCv6 len 36
act UPD ST REQ count 1
id:  creatorid: 

16:29:33.604120 10.240.252.77  10.240.252.78: PFSYNCv6 len 544
act BULK UPD STAT count 1
creatorid: b33d7f45 age: 00:00:00 status: start

18:42:57.518630 10.240.252.78  10.240.252.77: PFSYNCv6 len 124
act UPD ST REQ count 1
id:  creatorid: 

18:42:57.518634 10.240.252.77  10.240.252.78: PFSYNCv6 len 1384
act BULK UPD STAT count 1
creatorid: b33d7f45 age: 00:00:00 status: start

20:56:21.433270 10.240.252.78  10.240.252.77: PFSYNCv6 len 208
act UPD ST REQ count 1
id:  creatorid: 

20:56:21.433283 10.240.252.77  10.240.252.78: PFSYNCv6 len 628
act BULK UPD STAT count 1
creatorid: b33d7f45 age: 00:00:00 status: start

23:09:45.347531 10.240.252.78  10.240.252.77: PFSYNCv6 len 36
act UPD ST REQ count 1
id:  creatorid: 

23:09:45.347534 10.240.252.77  10.240.252.78: PFSYNCv6 len 292
act BULK UPD STAT count 1
creatorid: b33d7f45 age: 00:00:00 status: start

01:23:09.262083 10.240.252.78  10.240.252.77: PFSYNCv6 len 36
act UPD ST REQ count 1
id:  creatorid: 

01:23:09.262093 10.240.252.77  10.240.252.78: PFSYNCv6 len 712
act BULK UPD STAT count 1
creatorid: b33d7f45 age: 00:00:00 status: start

03:36:33.176294 10.240.252.78  10.240.252.77: PFSYNCv6 len 616
act UPD ST REQ count 1
id:  creatorid: 

03:36:33.176300 10.240.252.77  10.240.252.78: PFSYNCv6 len 628
act BULK UPD STAT count 1
creatorid: b33d7f45 age: 00:00:00 status: start

05:49:57.090125 10.240.252.78  10.240.252.77: PFSYNCv6 len 124
act UPD ST REQ count 1
id:  creatorid: 

05:49:57.090130 10.240.252.77  10.240.252.78: PFSYNCv6 len 1132
act BULK UPD STAT count 1
creatorid: b33d7f45 age: 00:00:00 status: start


/T





On Tue, Sep 2, 2014 at 12:07 PM, Tony Sarendal t...@polarcap.org wrote:

 As Chuck pointed out this has nothing to do with pfsense or freebsd.

 While I dig deeper I'm running with the following config to get around the
 problem:
 pf1.swe1# cat /etc/hostname.pfsync0
 ! sleep 10
 ! ifconfig $1 syncdev vlan44 syncpeer 10.240.252.77 up

 pf1.swe1#

 I see the request for the bulk transfer now, and the bulk transfer
 starting.
 Although bulk transfer performance looks like a problem, but that is for
 another thread.

 /T



 On Sat, Aug 30, 2014 at 9:31 PM, System Administrator ad...@bitwise.net
 wrote:

 And what does OP's message have to do with pfSense ??? (especially
 since he's clearly indicating currently supported OpenBSD versions 5.4
 and 5.5 near the bottom...)

 On 30 Aug 2014 at 14:22, Chuck Burns wrote:

  On Saturday, August 30, 2014 8:27:24 AM Tony Sarendal wrote:
   Good morning,
  
   I'm having issues with pfsync on trunk interfaces, although I suspect
   it to
  snip
   Running on pfsync on trunk(4) that initial request never shows up, and
   the bulk update never starts/finishes. I would like to run pfsync on
   trunk(4) lacp link, but as it looks now I have firewalls with carp
   demote counter 33 forever.
  snip
 
  pfSense is FreeBSD-based. not OpenBSD-based...
 
  different versions of pf between OpenBSD and FreeBSD
 
  --
  Chuck Burns
  Audemus Jura Nostra Defendere

pfsync and trunk

[Tony Sarendal]

Good morning,

I'm having issues with pfsync on trunk interfaces, although I suspect it to
be
any interface that is slow to start. When I run pfsync on a vlan interface
on a trunk(4),
the pfsync bulk transfer never completes.

Running pfsync on an interface that starts quickly I see:
07:41:45.982402 arp who-has 10.240.252.2 tell 10.240.252.2
07:41:45.982517 10.240.252.2: PFSYNCv6 len 36
act UPD ST REQ count 1
id:  creatorid: 
 (DF) [tos 0x10]
07:41:45.982606 10.240.252.1: PFSYNCv6 len 36
act BULK UPD STAT count 1
creatorid: e9bd40d6 age: 00:00:00 status: start
 (DF) [tos 0x10]
...snip...
07:41:46.062065 10.240.252.1: PFSYNCv6 len 304
act BULK UPD STAT count 1
creatorid: e9bd40d6 age: 00:00:01 status: end
act UPD ST count 1
...
 (DF) [tos 0x10]


Running on pfsync on trunk(4) that initial request never shows up, and the
bulk update never starts/finishes. I would like to run pfsync on trunk(4)
lacp link, but as it looks now I have firewalls with carp demote counter 33
forever.

Anyone else having problems with this ? Anything I can do to improve the
situation ?

Tested with 5.4 and 5.5, real and virtual machines, failover and lacp
trunk(4).

Regards Tony

Re: Does OpenBGPd suffer collateral damage with this?

[Tony Sarendal]

What a horrible article. I thought the kebab I just had for lunch ruined my
day, reading that was worse.


On Mon, Aug 18, 2014 at 2:27 AM, Rod Whitworth glis...@witworx.com wrote:

 http://www.smh.com.au/technology/technology-news/how-flakey-is-the-inter
 net-20140816-104t8p.html

 I would love to hear that our beloved BGP routers are the only ones
 that don't get screwed or at least we are one of the few.

 I haven't heard any noises from the hosting site that I look after.


 *** NOTE *** Please DO NOT CC me. I am subscribed to the list.
 Mail to the sender address that does not originate at the list server is
 tarpitted. The reply-to: address is provided for those who feel compelled
 to reply off list. Thankyou.

 Rod/
 ---
 This life is not the real thing.
 It is not even in Beta.
 If it was, then OpenBSD would already have a man page for it.

Re: Does OpenBGPd suffer collateral damage with this?

[Tony Sarendal]

Let me clarify the reason I thought the article was horrible. Highlights:

...
There are very few experts in this. It really is the deep magic,
...
Unfortunately, as well as the 4.2 billion IP address limit, the internet is
bound by another arbitrary restriction: the 512,000 slots in the BGP grid.
...
To fix it, they need to reboot the routers, and lots of them will be old
machines that have never been rebooted before, and sometimes when you
reboot something like that it doesn't switch back on again,
...


Regards T




On Mon, Aug 18, 2014 at 1:26 PM, Matthias Appel appel.matth...@gmail.com
wrote:

  -Ursprüngliche Nachricht-
  Von: owner-m...@openbsd.org [mailto:owner-m...@openbsd.org] Im
  Auftrag von Tony Sarendal
  Gesendet: Montag, 18. August 2014 12:55
  An: misc
  Betreff: Re: Does OpenBGPd suffer collateral damage with this?
 
  What a horrible article. I thought the kebab I just had for lunch ruined
 my
  day, reading that was worse.
 

 If this ruined your day, please refrain from reading further articles about
 BGP.

 If you want to experience _real_ pain, read
 http://www.bgpmon.net/chinese-isp-hijacked-10-of-the-internet/

 BR,

 Matthias

routes stuck in bgpd after ifconfig destroy

[Tony Sarendal]

Tested on 5.2 and current.
routes get stuck in bgpd after ifconfig destroy.

titan# cat /etc/bgpd.conf


AS 65001
router-id 10.1.1.1
network inet connected
network inet static

titan# bgpctl show rib

flags: * = Valid,  = Selected, I = via IBGP, A = Announced, S = Stale
origin: i = IGP, e = EGP, ? = Incomplete

flags destination  gateway  lpref   med aspath origin
AI*  10.1.1.0/24  0.0.0.0100 0 i
AI*  172.29.1.0/240.0.0.0100 0 i
titan# ifconfig vlan102 create

titan# ifconfig vlan102 vlandev em1 vlan 102 up

titan# ifconfig vlan102 192.168.1.1/24

titan# route add 192.0.2.1/32 192.168.1.100
add host 192.0.2.1/32: gateway 192.168.1.100
titan# bgpctl show rib

flags: * = Valid,  = Selected, I = via IBGP, A = Announced, S = Stale
origin: i = IGP, e = EGP, ? = Incomplete

flags destination  gateway  lpref   med aspath origin
AI*  10.1.1.0/24  0.0.0.0100 0 i
AI*  172.29.1.0/240.0.0.0100 0 i
AI*  192.0.2.1/32 0.0.0.0100 0 i
AI*  192.168.1.0/24   0.0.0.0100 0 i
titan# ifconfig vlan102 destroy

titan# bgpctl show rib
flags: * = Valid,  = Selected, I = via IBGP, A = Announced, S = Stale
origin: i = IGP, e = EGP, ? = Incomplete

flags destination  gateway  lpref   med aspath origin
AI*  10.1.1.0/24  0.0.0.0100 0 i
AI*  172.29.1.0/240.0.0.0100 0 i
AI*  192.0.2.1/32 0.0.0.0100 0 i
titan# bgpctl show fib | grep 192.0.2.1
*S   8 192.0.2.1/32 192.168.1.100
titan# netstat -rn | grep 192.0.2.1
titan#

/T

Re: Intel E3-1270 and AES-NI

[Tony Sarendal]

On Tue, Apr 3, 2012 at 10:49 PM, mxb m...@alumni.chalmers.se wrote:


 On Apr 3, 2012, at 4:31 PM, Tony Sarendal wrote:

  On Tue, Apr 3, 2012 at 3:41 PM, Jonathan Gray j...@jsg.id.au wrote:
 
  On Tue, Apr 03, 2012 at 03:09:37PM +0200, Tony Sarendal wrote:
  When testing new boxes with Intel E3-1270 cpu I don't see AES on the
  cpu's
  in dmesg.
  Does this mean that the aes-ni stuff isn't used on these ? I was a bit
  curious to see if it had any effect on ipsec performance.
 
  According to
 
 
 http://ark.intel.com/products/52276/Intel-Xeon-Processor-E3-1270-%288M-Cache-3_40-GHz%29
 
  it does support it.  So it sounds like a problem with the bios.  It
 would
  be printing along with the other cpuid flags in the cpu part
  of dmesg were it enabled.  And if the cpuid says it is not present,
  it is not used.
 
 
  You are star. It was disabled in bios.
 
  Cheers.
 

 Sometimes you even need to flash BIOS to have it.


Worked fine here. Performance boost depended a lot on packet size, a full
speed one direction tcp data transfer
got a 30% boost from enabling aes-ni. Small packet size, 200 byte mtu in
sending direction, gave around 5% boost.

The test box has been doing 400Mbps of large frame data transfer for a day
or so now.

One interesting thing was that running with SP kernel two low-latency,
high-speed, tcp tranfers could
starve userland badly enough to drop bgp sessions where as with MP kernel
the box remained responsive
no matter how many tcp sessions I shot through it.

/T

Intel E3-1270 and AES-NI

[Tony Sarendal]

When testing new boxes with Intel E3-1270 cpu I don't see AES on the cpu's
in dmesg.
Does this mean that the aes-ni stuff isn't used on these ? I was a bit
curious to see if it had any effect on ipsec performance.

Regards Tony

test3.pio# dmesg
OpenBSD 5.1-current (GENERIC.MP) #258: Mon Apr  2 12:25:25 MDT 2012
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 8566009856 (8169MB)
avail mem = 8315633664 (7930MB)
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xeadc0 (107 entries)
bios0: vendor American Megatrends Inc. version 1.1a date 09/28/2011
bios0: Supermicro X9SCI/X9SCA
acpi0 at bios0: rev 2
acpi0: sleep states S0 S1 S4 S5
acpi0: tables DSDT FACP APIC SSDT MCFG HPET SPMI EINJ ERST HEST BERT
acpi0: wakeup devices PS2K(S1) PS2M(S1) UAR1(S4) UAR2(S4) BR20(S1) EUSB(S4)
USBE(S4) PEX4(S4) PEX5(S4) PEX6(S4) PEX7(S4) GBE_(S4) P0P1(S4) P0P2(S4)
P0P3(S4) P0P4(S4) PEX0(S4) SLPB(S0) PWRB(S4)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(R) CPU E31270 @ 3.40GHz, 3392.78 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,PCLMUL,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,x2APIC,POPCNT,XSAVE,AVX,NXE,LONG,LAHF
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: apic clock running at 99MHz
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Xeon(R) CPU E31270 @ 3.40GHz, 3392.30 MHz
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,PCLMUL,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,x2APIC,POPCNT,XSAVE,AVX,NXE,LONG,LAHF
cpu1: 256KB 64b/line 8-way L2 cache
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Xeon(R) CPU E31270 @ 3.40GHz, 3392.30 MHz
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,PCLMUL,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,x2APIC,POPCNT,XSAVE,AVX,NXE,LONG,LAHF
cpu2: 256KB 64b/line 8-way L2 cache
cpu3 at mainbus0: apid 6 (application processor)
cpu3: Intel(R) Xeon(R) CPU E31270 @ 3.40GHz, 3392.30 MHz
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,PCLMUL,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,x2APIC,POPCNT,XSAVE,AVX,NXE,LONG,LAHF
cpu3: 256KB 64b/line 8-way L2 cache
cpu4 at mainbus0: apid 1 (application processor)
cpu4: Intel(R) Xeon(R) CPU E31270 @ 3.40GHz, 3392.30 MHz
cpu4:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,PCLMUL,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,x2APIC,POPCNT,XSAVE,AVX,NXE,LONG,LAHF
cpu4: 256KB 64b/line 8-way L2 cache
cpu5 at mainbus0: apid 3 (application processor)
cpu5: Intel(R) Xeon(R) CPU E31270 @ 3.40GHz, 3392.30 MHz
cpu5:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,PCLMUL,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,x2APIC,POPCNT,XSAVE,AVX,NXE,LONG,LAHF
cpu5: 256KB 64b/line 8-way L2 cache
cpu6 at mainbus0: apid 5 (application processor)
cpu6: Intel(R) Xeon(R) CPU E31270 @ 3.40GHz, 3392.30 MHz
cpu6:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,PCLMUL,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,x2APIC,POPCNT,XSAVE,AVX,NXE,LONG,LAHF
cpu6: 256KB 64b/line 8-way L2 cache
cpu7 at mainbus0: apid 7 (application processor)
cpu7: Intel(R) Xeon(R) CPU E31270 @ 3.40GHz, 3392.30 MHz
cpu7:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,PCLMUL,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,SSE4.1,SSE4.2,x2APIC,POPCNT,XSAVE,AVX,NXE,LONG,LAHF
cpu7: 256KB 64b/line 8-way L2 cache
ioapic0 at mainbus0: apid 0 pa 0xfec0, version 20, 24 pins
acpimcfg0 at acpi0 addr 0xe000, bus 0-255
acpihpet0 at acpi0: 14318179 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 7 (BR20)
acpiprt2 at acpi0: bus 3 (PEX4)
acpiprt3 at acpi0: bus 4 (PEX5)
acpiprt4 at acpi0: bus 5 (PEX6)
acpiprt5 at acpi0: bus 6 (PEX7)
acpiprt6 at acpi0: bus 1 (P0P1)
acpiprt7 at acpi0: bus -1 (P0P2)
acpiprt8 at acpi0: bus -1 (P0P3)
acpiprt9 at acpi0: bus -1 (P0P4)
acpiprt10 at acpi0: bus 2 (PEX0)
acpicpu0 at acpi0: C3, C1, PSS
acpicpu1 at acpi0: C3, C1, PSS
acpicpu2 at acpi0: C3, C1, PSS
acpicpu3 at acpi0: C3, C1, PSS
acpicpu4 at acpi0: C3, C1, PSS
acpicpu5 at acpi0: C3, C1, PSS
acpicpu6 at acpi0: C3, C1, PSS
acpicpu7 at acpi0: C3, C1, PSS
acpibtn0 at acpi0: SLPB
acpibtn1 at acpi0: PWRB
ipmi at mainbus0 not configured
cpu0: Enhanced SpeedStep 3392 MHz: speeds: 3401, 3400, 3200, 3000, 

Re: Intel E3-1270 and AES-NI

[Tony Sarendal]

On Tue, Apr 3, 2012 at 3:41 PM, Jonathan Gray j...@jsg.id.au wrote:

 On Tue, Apr 03, 2012 at 03:09:37PM +0200, Tony Sarendal wrote:
  When testing new boxes with Intel E3-1270 cpu I don't see AES on the
 cpu's
  in dmesg.
  Does this mean that the aes-ni stuff isn't used on these ? I was a bit
  curious to see if it had any effect on ipsec performance.

 According to

 http://ark.intel.com/products/52276/Intel-Xeon-Processor-E3-1270-%288M-Cache-3_40-GHz%29

 it does support it.  So it sounds like a problem with the bios.  It would
 be printing along with the other cpuid flags in the cpu part
 of dmesg were it enabled.  And if the cpuid says it is not present,
 it is not used.


You are star. It was disabled in bios.

Cheers.

4.9, set reassemble no + block log + fragments = panic

[Tony Sarendal]

Good evening,

the last two days we have experienced panics sequentially across all of our
peering boxes.
After one day of coffee, thinking and reading, I found this in 4.9. (5.0+
looks good):

target49# ifconfig vlan69
vlan69: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST mtu 1500
lladdr 00:0c:29:38:f3:c5
priority: 0
vlan: 69 priority: 0 parent interface: em1
groups: vlan
status: active
inet6 fe80::20c:29ff:fe38:f3c5%vlan69 prefixlen 64 scopeid 0x5
inet 192.168.69.49 netmask 0xff00 broadcast 192.168.69.255
target49# cat /etc/pf.conf

set skip on lo
set reassemble no

block in log quick

target49#

sender51# tcpdump -n -i vlan69 -v
tcpdump: listening on vlan69, link-type EN10MB
tcpdump: WARNING: compensating for unaligned libpcap packets
20:55:58.958739 192.168.69.1.10562  192.168.69.49.1234: udp 2000 (frag
58745:1480@0+) (ttl 64, len 1500)
20:55:58.958745 192.168.69.1  192.168.69.49: (frag 58745:528@1480) (ttl
64, len 548)
^C

Mar 20 20:57:17 target49 /bsd: uvm_fault(0x80d1b0e0, 0x0, 0, 1) - e
Mar 20 20:57:17 target49 /bsd: fatal page fault in supervisor mode
Mar 20 20:57:17 target49 /bsd: trap type 6 code 0 rip 80245557 cs 8
rflags 10246 cr2  0 cpl 5 rsp 8bba0b40
Mar 20 20:57:17 target49 /bsd: panic: trap type 6, code=0,
pc=80245557
Mar 20 20:57:17 target49 /bsd: Starting stack trace...
Mar 20 20:57:17 target49 /bsd: panic() at panic+0xf5
Mar 20 20:57:17 target49 /bsd: trap() at trap+0x6fd
Mar 20 20:57:17 target49 /bsd: --- trap (number 6) ---
Mar 20 20:57:17 target49 /bsd: pf_change_ap() at pf_change_ap+0x57
Mar 20 20:57:18 target49 /bsd: pf_translate() at pf_translate+0x27d
Mar 20 20:57:18 target49 /bsd: pflog_bpfcopy() at pflog_bpfcopy+0x233
Mar 20 20:57:18 target49 /bsd: bpf_catchpacket() at bpf_catchpacket+0xd8
Mar 20 20:57:18 target49 /bsd: bpf_mtap_pflog() at bpf_mtap_pflog+0x8f
Mar 20 20:57:18 target49 /bsd: pflog_packet() at pflog_packet+0x223
Mar 20 20:57:18 target49 /bsd: pf_test_fragment() at pf_test_fragment+0x502
Mar 20 20:57:18 target49 /bsd: pf_test() at pf_test+0x7ef
Mar 20 20:57:18 target49 /bsd: ipv4_input() at ipv4_input+0x22a
Mar 20 20:57:18 target49 /bsd: ipintr() at ipintr+0x51
Mar 20 20:57:18 target49 /bsd: netintr() at netintr+0xda
Mar 20 20:57:18 target49 /bsd: softintr_dispatch() at softintr_dispatch+0x5d
Mar 20 20:57:18 target49 /bsd: Xsoftnet() at Xsoftnet+0x28
Mar 20 20:57:18 target49 /bsd: --- interrupt ---
Mar 20 20:57:18 target49 /bsd: end of kernel
Mar 20 20:57:18 target49 /bsd: end trace frame: 0x6a7, count: 242
Mar 20 20:57:18 target49 /bsd: 0x8:
Mar 20 20:57:18 target49 /bsd: End of stack trace.
Mar 20 20:57:18 target49 /bsd: End of stack trace.
Mar 20 20:57:18 target49 /bsd: dump to dev 4,1 not possible
Mar 20 20:57:18 target49 /bsd: rebooting...

Ignore the timestamps, the box panics immediately after getting the
fragmented packet.
I could reproduce it on vmware 4.9-stable with GENERIC/GENERIC.MP, so no
dmesg attached.

Regards Tony

Re: bgpctl shiw rib out displaying incorrect information

[Tony Sarendal]

On Fri, Sep 16, 2011 at 2:34 PM, Claudio Jeker cje...@diehard.n-r-g.comwrote:

 On Wed, Aug 31, 2011 at 04:37:49PM +0200, Tony Sarendal wrote:
  On Wed, Aug 31, 2011 at 4:24 PM, Josh Hoppes josh.hop...@gmail.com
 wrote:
 
   Why are you using set nexthop self and then trying to change that
   with the filter allow quick to 172.29.1.52 set nexthop 172.29.1.200.
   If you don't want your nexthop to be yourself don't tell bgpd to do
   that.
  
  
  To show a bug in bgpctl/bgpd (or where ever it may be).
  Dont you want to be able to trust the information bgpctl gives you ?
 

 Yes there is a bug in bgpd. The problem is that set nexthop self was
 sticky and overriding the other set nexthop that came after.

 The following diff should solve those issues.
 --
 :wq Claudio


Looks fine when I apply it to the test boxes.
Cheers.

/Tony

Re: bgpctl shiw rib out displaying incorrect information

[Tony Sarendal]

On Wed, Aug 31, 2011 at 9:51 AM, Patrick Lamaiziere
patf...@davenulle.orgwrote:

 Le Wed, 31 Aug 2011 07:19:15 +0200,
 Tony Sarendal t...@polarcap.org a icrit :

 Hi,

  current1# cat /etc/bgpd.conf
  AS 65001
  network 10.0.1.0/24
 
  current1# bgpctl show rib nei 172.29.1.52 out
  flags: * = Valid,  = Selected, I = via IBGP, A = Announced
  origin: i = IGP, e = EGP, ? = Incomplete
 
  flags destination  gateway  lpref   med aspath origin
  AI*  10.0.1.0/24  172.29.1.200   100 0 i

 So you announce (A) via IBGP (I) the route 10.0.1.0/24, looks good no?.

  current2# bgpctl show rib nei 172.29.1.51 in
  flags: * = Valid,  = Selected, I = via IBGP, A = Announced
  origin: i = IGP, e = EGP, ? = Incomplete
 
  flags destination  gateway  lpref   med aspath origin
  I*   10.0.1.0/24  172.29.1.51100 0 i

 And you receive the route via IBGP (I), looks good too.

 Where is the problem?


Sender says next hop = 172.29.1.100, receiver says .51.
show rib out in this case shows incorrect nexthop.

Regards Tony

Re: bgpctl shiw rib out displaying incorrect information

[Tony Sarendal]

On Wed, Aug 31, 2011 at 11:01 AM, Andre Keller a...@list.ak.cx wrote:

 Hi

 Am 31.08.2011 10:23, schrieb Tony Sarendal:
  Sender says next hop = 172.29.1.100, receiver says .51.
  show rib out in this case shows incorrect nexthop.

 Well thats kind of the point of having set nexthop self in the config...


You are missing the point, completely.
bgpctl show rib out displays incorrect information.

Regards Tony

Re: bgpctl shiw rib out displaying incorrect information

[Tony Sarendal]

On Wed, Aug 31, 2011 at 4:24 PM, Josh Hoppes josh.hop...@gmail.com wrote:

 Why are you using set nexthop self and then trying to change that
 with the filter allow quick to 172.29.1.52 set nexthop 172.29.1.200.
 If you don't want your nexthop to be yourself don't tell bgpd to do
 that.


To show a bug in bgpctl/bgpd (or where ever it may be).
Dont you want to be able to trust the information bgpctl gives you ?

Regards Tony

bgpctl shiw rib out displaying incorrect information

[Tony Sarendal]

current1# cat /etc/bgpd.conf
AS 65001
network 10.0.1.0/24

neighbor 172.29.1.52 {
remote-as 65001
set nexthop self
descr current2
local-address 172.29.1.51
}

allow quick to 172.29.1.52 set nexthop 172.29.1.200
allow to any
allow from any

current1# bgpctl show rib nei 172.29.1.52 out
flags: * = Valid,  = Selected, I = via IBGP, A = Announced
origin: i = IGP, e = EGP, ? = Incomplete

flags destination  gateway  lpref   med aspath origin
AI*  10.0.1.0/24  172.29.1.200   100 0 i
current1#



current2# bgpctl show rib nei 172.29.1.51 in
flags: * = Valid,  = Selected, I = via IBGP, A = Announced
origin: i = IGP, e = EGP, ? = Incomplete

flags destination  gateway  lpref   med aspath origin
I*   10.0.1.0/24  172.29.1.51100 0 i
current2#
current2# cat /etc/bgpd.conf
AS 65001
network 10.0.2.0/24

neighbor 172.29.1.51 {
remote-as 65001
set nexthop self
local-address 172.29.1.52
descr current1
}

allow to any
allow from any

Tested on -current, see the same on 4.9.

Regards Tony

Re: isakmpd and INVALID_COOKIE

[Tony Sarendal]

On Mon, Jul 4, 2011 at 4:12 PM, rancor theran...@gmail.com wrote:

 Ah =) Thanks!

 // rancor

 2011/7/4 Stuart Henderson s...@spacehopper.org:
   On 2011-07-02, rancor theran...@gmail.com wrote:
  Hi.
 
  I have two separate ipsec tunnels from 4.9 boxes and both are
  generating this message i /var/log/messages once every hour or two
  Jul  2 08:14:54 hostname isakmpd[28247]: message_recv: invalid
  cookie(s) 576scrambled03c2
  Jul  2 08:14:54 hostname isakmpd[28247]: dropped message from
  x.x.x.x port 500 due to notification type INVALID_COOKIE
 
  The tunnels works perfect but I still wounder why I got this message.
 
  This is my ipsec.conf on host x
  ike esp transport from x.x.x.x to y.y.y.y psk scrambled
 
  and on host y
  ike esp transport from y.y.y.y to x.x.x.x psk scrambled
 
  Any idea?
 
  Best regards rancor
 
 
 
  If you're running isakmpd from 4.8 or 4.9 with IKE you want to pull
  up src/sbin/isakmpd/dh.c to r1.14 otherwise you will certainly
  see problems from time to time.


Is this a cosmetic thing or does it affect connectivity ?

We are having issues with gaps in connectivity on our ipsec links with a
basic ike setup,
an issue we're starting to look into now.

Regards Tony

Re: isakmpd and INVALID_COOKIE

[Tony Sarendal]

On Fri, Jul 8, 2011 at 4:09 PM, Stuart Henderson s...@spacehopper.orgwrote:

 On 2011-07-08, Tony Sarendal t...@polarcap.org wrote:
   If you're running isakmpd from 4.8 or 4.9 with IKE you want to pull
   up src/sbin/isakmpd/dh.c to r1.14 otherwise you will certainly
   see problems from time to time.
 
 
  Is this a cosmetic thing or does it affect connectivity ?

 dh.c r1.14 affects stability. Between 4.7 and 4.8 isakmpd switched
 from internal to openssl DH; an openssl function wasn't padding with
 leading 0's where it was expected that they would, so there was junk
 at the end of the key, causing key mismatches.

Sounds like a candidate to our issues that we are seeing on both 4.8 and
4.9.
We see it quite easily as we run gre tunnels with bgp inside them using
ipsec
to encrypt gre.

We are seeing the connectivity issue antyhing from a few times a day to a
few times a week.
And the time I caught it while it was going on things started working
immediately after some
bi-directional ike traffic.

Regards Tony

redistributing routes

[Tony Sarendal]

Is there a way to redistribute routes from BGP to OSPF using bgpd and ospfd
?

I have a network where the core concists of openbsd devices using bgpd to
distribute
routing information. At present we need to use static routing if we connect
devices that
do not support BGP.

Regards Tony

Re: redistributing routes

[Tony Sarendal]

On Sat, Oct 23, 2010 at 2:05 PM, Insan Praja SW insan.pr...@gmail.comwrote:

 Hi Tony,

 On Sat, 23 Oct 2010 18:44:46 +0700, Tony Sarendal t...@polarcap.org
 wrote:

 Is there a way to redistribute routes from BGP to OSPF using bgpd and ospfd
 ?


 on bgpd.conf you might want to do this:

 match from $peer1 inet prefix xxx.xxx.xxx.xxx/xx prefixlen bla_bla set
 rtlabel from_bgpd

 on ospfd.conf you do this:

 redistribute rtlabel from_bgpd


 I have a network where the core concists of openbsd devices using bgpd to
 distribute
 routing information. At present we need to use static routing if we
 connect
 devices that
 do not support BGP.

 Regards Tony


 Good Luck,




I was considering an approach like that, but the bgpd man page suggests that
it wouldnt work.

ATTRIBUTE SET
 AS path attributes can be modified with set.
 set can be used on network statements, in neighbor or group blocks, and
 on filter rules.  Attribute sets can be expressed as lists.
 The following attributes can be modified:
...
 rtlabel label
 Add the prefix with the specified label to the kernel routing
 table.


Is this an error in the page or me reading it wrong ?
If this works as expected, is this the recommended way of doing it ?


Regards Tony

Re: redistributing routes

[Tony Sarendal]

On Sat, Oct 23, 2010 at 3:07 PM, Henning Brauer lists-open...@bsws.dewrote:

 * Tony Sarendal t...@polarcap.org [2010-10-23 14:29]:
   rtlabel label
   Add the prefix with the specified label to the kernel
 routing
   table.
 
  Is this an error in the page or me reading it wrong ?

 debatable... this could be worded better. with rtlabel foo, bgpd will
 add the label foo to all routes it inserts.

  If this works as expected, is this the recommended way of doing it ?

 i don't see anything wrong with that approach.




Very good. Thanks.

Regards Tony

Re: redistributing routes

[Tony Sarendal]

On Sat, Oct 23, 2010 at 6:16 PM, Stuart Henderson s...@spacehopper.orgwrote:

 On 2010-10-23, Tony Sarendal t...@polarcap.org wrote:
   rtlabel label
   Add the prefix with the specified label to the kernel
 routing
   table.

 I think this should be:

 Add the prefix to the kernel routing table with the specified label.

 Index: bgpd.conf.5
 ===
 RCS file: /cvs/src/usr.sbin/bgpd/bgpd.conf.5,v
 retrieving revision 1.112
 diff -u -p -r1.112 bgpd.conf.5
 --- bgpd.conf.5 13 Oct 2010 21:04:13 -  1.112
 +++ bgpd.conf.5 23 Oct 2010 16:12:36 -
 @@ -1432,9 +1432,9 @@ times to the
  .Em AS path .
  .Pp
  .It Ic rtlabel Ar label
 -Add the prefix with the specified
 -.Ar label
 -to the kernel routing table.
 +Add the prefix to the kernel routing table
 +with the specified
 +.Ar label .
  .Pp
  .It Ic weight Ar number
  The


 ...maybe we could also add something like, Can be used to
 redistribute routes to another routing protocol daemon,
 or maybe we should leave that for people to figure out themselves.


How does OpenBSD handle the same prefix being in both bgpd and ospfd ?

I connect devices to the core network using two core routers and
redistributing
BGP-OSPF would be happening on both of them.

Regards Tony

Re: redistributing routes

[Tony Sarendal]

On Sat, Oct 23, 2010 at 8:02 PM, Henning Brauer lists-open...@bsws.dewrote:

 * Tony Sarendal t...@polarcap.org [2010-10-23 19:03]:
  How does OpenBSD handle the same prefix being in both bgpd and ospfd ?

 in general? OSPF routes have priority over BGP routes. that's
 implemented kernel routing table side and the daemons setting the
 priority field to their respective priorities when inserting their
 routes.


Does this mean that bgpd and ospfd can happily co-exist on the same box ?

As an example:
Prefix A shows up in BGP, later it shows up in OSPF,
even later it is withdrawn from OSPF. Will the prefix in BGP now be in the
fib ?

OSPF being the winner is not optimal in my case, but being predictable
is good enough.

 I connect devices to the core network using two core routers and
  redistributing
  BGP-OSPF would be happening on both of them.

 that I dunno OTOH


Being able to redist BGP-OSPF and not connecting ospfd to the fib would
do what I want. Unfortunately the manpage for ospfd.conf doesn't seem to
support
this setup.

 fib-update (yes|no)
 If set to no, do not update the Forwarding Information Base,
 a.k.a. the kernel routing table.  The default is yes.  Setting
 fib-update to no will implicitly set the stub router option to
 ensure that no traffic tries to transit via this router.


Regards Tony

Re: redistributing routes

[Tony Sarendal]

On Sat, Oct 23, 2010 at 8:45 PM, Tony Sarendal t...@polarcap.org wrote:



 On Sat, Oct 23, 2010 at 8:02 PM, Henning Brauer lists-open...@bsws.dewrote:

 * Tony Sarendal t...@polarcap.org [2010-10-23 19:03]:
  How does OpenBSD handle the same prefix being in both bgpd and ospfd ?

 in general? OSPF routes have priority over BGP routes. that's
 implemented kernel routing table side and the daemons setting the
 priority field to their respective priorities when inserting their
 routes.


 Does this mean that bgpd and ospfd can happily co-exist on the same box ?

 As an example:
 Prefix A shows up in BGP, later it shows up in OSPF,
 even later it is withdrawn from OSPF. Will the prefix in BGP now be in the
 fib ?

 OSPF being the winner is not optimal in my case, but being predictable
 is good enough.

   I connect devices to the core network using two core routers and
  redistributing
  BGP-OSPF would be happening on both of them.

 that I dunno OTOH


 Being able to redist BGP-OSPF and not connecting ospfd to the fib would
 do what I want. Unfortunately the manpage for ospfd.conf doesn't seem to
 support
 this setup.

  fib-update (yes|no)
  If set to no, do not update the Forwarding Information Base,
  a.k.a. the kernel routing table.  The default is yes.  Setting
  fib-update to no will implicitly set the stub router option to
  ensure that no traffic tries to transit via this router.


I mean this would do what I want if bgpd and ospfd can't co-exist.

Regards Tony

Re: problems with carp based firewall - all connections are suspended after falling back from failover

[Tony Sarendal]

On Sat, Apr 10, 2010 at 9:44 AM, tom baecker tb4...@googlemail.com wrote:

 Hello,

 I've setup a openbsd-ha firewall, based on the
 http://www.openbsd.org/faq/pf/carp.html.

 If the master goes down - the backup system become the Master rule.
 All established connections are in sync and stay active - so thats
 perfect.
 But if the original Master system comes back again and fall back to
 the Master state - all established connections are broken, maybe they
 not successfully synced to the old master?

 Is there a way to prevent fallback, so the backup system stay in
 Master rule after failover?
 Maybe also I've a wrong setup.

 Primary setup:
 /etc/hostname.carp0:
 inet 10.1.1.1 255.255.255.0 10.100.255.255 vhid 1 pass bbb
 /etc/hostname.carp1:
 inet 10.1.2.1 255.255.255.0 10.68.255.255 vhid 2 pass aaa
 /etc/hostname.carp2:
 inet 10.1.3.1 255.255.255.0 10.101.10.255 vhid 3 pass xxx
 /etc/hostname.pfsync0
 up syncdev em1

 net.inet.carp.preempt=1
 net.inet.ip.forwarding=1
 net.inet.carp.log=7

 pf.conf
 # allow pfsync
 pass quick on em1 proto pfsync
 # allow carp
 pass quick on { em0, em2, em3 } proto carp keep state


 Standby setup:
 /etc/hostname.carp0:
 inet 10.1.1.1 255.255.255.0 10.100.255.255 vhid 1 advskew 100 pass bbb
 /etc/hostname.carp1:
 inet 10.1.2.1 255.255.255.0 10.68.255.255 vhid 2 advskew 100 pass aaa
 /etc/hostname.carp2:
 inet 10.1.3.1 255.255.255.0 10.101.10.255 vhid 3 advskew 100 pass xxx
 /etc/hostname.pfsync0
 up syncdev em1

 net.inet.carp.preempt=1
 net.inet.ip.forwarding=1
 net.inet.carp.log=7

 pf.conf
 # allow pfsync
 pass quick on em1 proto pfsync
 # allow carp
 pass quick on { em0, em2, em3 } proto carp keep state




 A failover and fallback gives me the follow entrys in the message log:

 the master goes down:
 Apr  9 16:02:05 fw-bkp /bsd: carp1: state transition: BACKUP - MASTER
 Apr  9 16:02:05 fw-bkp /bsd: carp0: state transition: BACKUP - MASTER
 Apr  9 16:02:05 fw-bkp /bsd: carp2: state transition: BACKUP - MASTER
 the master comes back:
 Apr  9 16:25:07 fw-bkp /bsd: carp0: state transition: MASTER - BACKUP
 Apr  9 16:25:07 fw-bkp /bsd: carp2: state transition: MASTER - BACKUP
 Apr  9 16:25:17 fw-bkp /bsd: carp1: state transition: MASTER - BACKUP


 the primary booting up and takeover the master rule:
 Apr  9 16:24:11 fw-pri /bsd: carp: carp0 demoted group carp to 129
 Apr  9 16:24:11 fw-pri /bsd: carp: carp1 demoted group carp to 130
 Apr  9 16:24:11 fw-pri /bsd: carp: carp2 demoted group carp to 131
 Apr  9 16:24:11 fw-pri /bsd: carp0: state transition: INIT - BACKUP
 Apr  9 16:24:11 fw-pri /bsd: carp: carp0 demoted group carp to 134
 Apr  9 16:24:12 fw-pri /bsd: carp: pfsync0 demoted group carp to 131
 Apr  9 16:24:12 fw-pri /bsd: carp: pfsync0 demoted group pfsync to 1
 Apr  9 16:24:12 fw-pri /bsd: carp1: state transition: INIT - BACKUP
 Apr  9 16:24:12 fw-pri /bsd: carp: carp1 demoted group carp to 130
 Apr  9 16:24:12 fw-pri /bsd: carp2: state transition: INIT - BACKUP
 Apr  9 16:24:12 fw-pri /bsd: carp: carp2 demoted group carp to 129
 Apr  9 16:24:12 fw-pri /bsd: carp1: state transition: BACKUP - MASTER
 Apr  9 16:24:29 fw-pri /bsd: carp: pfsync0 demoted group carp to 0
 Apr  9 16:24:29 fw-pri /bsd: carp: pfsync0 demoted group pfsync to 0
 Apr  9 16:24:30 fw-pri /bsd: carp0: state transition: BACKUP - MASTER
 Apr  9 16:24:30 fw-pri /bsd: carp2: state transition: BACKUP - MASTER


 hopefully you can help me.
 Regards,
 Tom


net.inet.carp.preempt   Allow virtual hosts to preempt each other.
Set it to 0 and give it a try.

/Tony

Re: unidentified system load

[Tony Sarendal]

On Sun, Mar 28, 2010 at 1:18 PM, Tony Sarendal t...@polarcap.org wrote:



 On Sun, Mar 28, 2010 at 10:41 AM, Mark Kettenis 
 mark.kette...@xs4all.nlwrote:

 It's worth trying to disable ichiic(4).


 Cheers, giving it a go on a few of them.


Over a week running with i386 4.6 and -current with ichiic(4) disabled.
The 6 boxes I updated all looking good.

Thanks Mark

Re: unidentified system load

[Tony Sarendal]

Is there a way to see where the cpu time is spent when it isn't in userland
?
I took one of our affected systems and killed everything on it as well as
disabling pf.

bmr1.brh# ps aux
USER   PID %CPU %MEM   VSZ   RSS TT  STAT  STARTED   TIME COMMAND
root 1  0.0  0.0   324   296 ??  Is 1Mar100:00.02 /sbin/init
root  8898  0.0  0.0   708  1200 ??  Is 1Mar100:00.02
/usr/sbin/sshd
root 29797  0.0  0.1  3424  2468 ??  Is 8:29AM0:00.11 sshd:
thehoff [priv] (sshd)
thehoff  27836  0.0  0.1  3396  1912 ??  S  8:29AM0:00.03 sshd:
theh...@ttyp0 (sshd)
thehoff   4730  0.0  0.0   480   408 p0  Is 8:29AM0:00.00 -ksh (ksh)
root 23806  0.0  0.0   476   460 p0  S  8:29AM0:00.01 -ksh (ksh)
root 15249  0.0  0.0   276   276 p0  R+/1   8:53AM0:00.00 ps -aux
root 25718  0.0  0.0   408   736 C0  Is+1Mar100:00.00
/usr/libexec/getty std.9600 ttyC0
root 30984  0.0  0.0   300   736 C1  Is+1Mar100:00.00
/usr/libexec/getty std.9600 ttyC1
root  7406  0.0  0.0   256   740 C2  Is+1Mar100:00.00
/usr/libexec/getty std.9600 ttyC2
root  1736  0.0  0.0   336   728 C3  Is+1Mar100:00.00
/usr/libexec/getty std.9600 ttyC3
root  1371  0.0  0.0   440   736 C5  Is+1Mar100:00.00
/usr/libexec/getty std.9600 ttyC5
bmr1.brh#

load averages:  0.08,  0.09,
0.08
08:52:43
12 processes:  11 idle, 1 on processor
CPU0 states:  0.0% user,  0.0% nice,  0.0% system,  0.2% interrupt, 99.8%
idle
CPU1 states:  0.0% user,  0.0% nice,  8.1% system,  0.0% interrupt, 91.9%
idle
Memory: Real: 5220K/351M act/tot  Free: 2916M  Swap: 0K/8197M used/tot

  PID USERNAME PRI NICE  SIZE   RES STATE WAIT  TIMECPU COMMAND
29797 root   20 3424K 2468K idle  netio 0:00  0.00% sshd
27836 thehoff20 3396K 1912K sleep/0   select0:00  0.00% sshd
1 root  100  324K  296K idle  wait  0:00  0.00% init
 8898 root   20  708K 1200K idle  select0:00  0.00% sshd
23806 root  180  476K  460K sleep/0   pause 0:00  0.00% ksh
32058 root  280  712K 1420K onproc/1  - 0:00  0.00% top
 4730 thehoff   180  480K  408K idle  pause 0:00  0.00% ksh
25718 root   30  408K  736K idle  ttyin 0:00  0.00% getty
 1736 root   30  336K  728K idle  ttyin 0:00  0.00% getty
30984 root   30  300K  736K idle  ttyin 0:00  0.00% getty
 1371 root   30  440K  736K idle  ttyin 0:00  0.00% getty
 7406 root   30  256K  740K idle  ttyin 0:00  0.00% getty

I suspect that there is some device in there that is being polled in some
funky way
as this only happens on these specific boxes. Even the fujitsu box marked as
kaputt with
red marker pen works as charm.

/T

Re: unidentified system load

[Tony Sarendal]

On Sun, Mar 28, 2010 at 10:41 AM, Mark Kettenis mark.kette...@xs4all.nlwrote:

 It's worth trying to disable ichiic(4).


Cheers, giving it a go on a few of them.

/Tony

unidentified system load

[Tony Sarendal]

I'm using supermicro boxes (dmesg below) as vpn routers. IPsec+gre+bgp.

After a few days uptime the boxes start reporting 8% system cpu, and at the
same time
they become unresponsive on the network approx every 10 seconds.
Any idea on how to find the reason for this is appreciated.
I have around 20 of these boxes running open and freebsd, so far all of the
openbsd boxes
display this behaviour using amd64, i386, sp and mp, 4.6 and various 4.7
snapshots.

I only see this on these specific supermicros. This happens on the devices
that don't
move any traffic as well.

Regards Tony

bmr0.mlt# ping -i 0.1 172.30.251.230
64 bytes from 172.30.251.230: icmp_seq=42 ttl=255 time=0.278 ms
64 bytes from 172.30.251.230: icmp_seq=43 ttl=255 time=0.328 ms
64 bytes from 172.30.251.230: icmp_seq=44 ttl=255 time=0.250 ms
64 bytes from 172.30.251.230: icmp_seq=45 ttl=255 time=402.911 ms
64 bytes from 172.30.251.230: icmp_seq=46 ttl=255 time=292.374 ms
64 bytes from 172.30.251.230: icmp_seq=47 ttl=255 time=181.836 ms
64 bytes from 172.30.251.230: icmp_seq=48 ttl=255 time=71.300 ms
64 bytes from 172.30.251.230: icmp_seq=49 ttl=255 time=0.255 ms
64 bytes from 172.30.251.230: icmp_seq=50 ttl=255 time=0.305 ms

bmr1.mlt# dmesg
OpenBSD 4.7-beta (GENERIC.MP) #427: Sun Feb 28 12:37:40 MST 2010
dera...@i386.openbsd.org:/usr/src/sys/arch/i386/compile/GENERIC.MP
cpu0: Intel(R) Core(TM)2 Duo CPU E8400 @ 3.00GHz (GenuineIntel 686-class)
3.01 GHz
cpu0:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,SMX,EST,TM2,CX16,xTPR
real mem  = 3487580160 (3326MB)
avail mem = 3391463424 (3234MB)
mainbus0 at root
bios0 at mainbus0: AT/286+ BIOS, date 07/24/09, BIOS32 rev. 0 @ 0xfdb70,
SMBIOS rev. 2.5 @ 0xcfedf000 (39 entries)
bios0: vendor Phoenix Technologies LTD version 1.30 date 07/24/2009
bios0: Supermicro X7SBi
acpi0 at bios0: rev 2
acpi0: tables DSDT FACP _MAR MCFG APIC BOOT SPCR ERST HEST BERT EINJ SLIC
SSDT SSDT SSDT SSDT SSDT SSDT SSDT SSDT SSDT
acpi0: wakeup devices PXHA(S5) PEX_(S5) LAN_(S5) USB4(S5) USB5(S5) USB7(S5)
ESB2(S5) EXP1(S5) EXP5(S5) EXP6(S5) USB1(S5) USB2(S5) USB3(S5) USB6(S5)
ESB1(S5) PCIB(S5) KBC0(S1) MSE0(S1) COM1(S5) COM2(S5) PWRB(S3)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: apic clock running at 333MHz
cpu1 at mainbus0: apid 1 (application processor)
cpu1: Intel(R) Core(TM)2 Duo CPU E8400 @ 3.00GHz (GenuineIntel 686-class)
3.01 GHz
cpu1:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,SMX,EST,TM2,CX16,xTPR
ioapic0 at mainbus0: apid 2 pa 0xfec0, version 20, 24 pins
ioapic1 at mainbus0: apid 3 pa 0xfecc, version 20, 24 pins
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 2 (PXHA)
acpiprt2 at acpi0: bus -1 (PEX_)
acpiprt3 at acpi0: bus 5 (EXP1)
acpiprt4 at acpi0: bus 13 (EXP5)
acpiprt5 at acpi0: bus 15 (EXP6)
acpiprt6 at acpi0: bus 17 (PCIB)
acpicpu0 at acpi0: C3, PSS
acpicpu1 at acpi0: C3, PSS
acpibtn0 at acpi0: PWRB
acpivideo0 at acpi0: IGD0
bios0: ROM list: 0xc/0x9000
ipmi at mainbus0 not configured
cpu0: Enhanced SpeedStep 3001 MHz: speeds: 3000, 2667, 2333, 2000 MHz
pci0 at mainbus0 bus 0: configuration mode 1 (bios)
pchb0 at pci0 dev 0 function 0 Intel 3200/3210 Host rev 0x01
ppb0 at pci0 dev 1 function 0 Intel 3200/3210 PCIE rev 0x01: apic 2 int 16
(irq 5)
pci1 at ppb0 bus 1
ppb1 at pci1 dev 0 function 0 Intel PCIE-PCIE rev 0x09
pci2 at ppb1 bus 2
Intel IOxAPIC rev 0x09 at pci1 dev 0 function 1 not configured
uhci0 at pci0 dev 26 function 0 Intel 82801I USB rev 0x02: apic 2 int 16
(irq 5)
uhci1 at pci0 dev 26 function 1 Intel 82801I USB rev 0x02: apic 2 int 17
(irq 10)
uhci2 at pci0 dev 26 function 2 Intel 82801I USB rev 0x02: apic 2 int 18
(irq 11)
ehci0 at pci0 dev 26 function 7 Intel 82801I USB rev 0x02: apic 2 int 18
(irq 11)
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 Intel EHCI root hub rev 2.00/1.00 addr 1
ppb2 at pci0 dev 28 function 0 Intel 82801I PCIE rev 0x02: apic 2 int 16
(irq 5)
pci3 at ppb2 bus 5
ppb3 at pci0 dev 28 function 4 Intel 82801I PCIE rev 0x02: apic 2 int 16
(irq 5)
pci4 at ppb3 bus 13
em0 at pci4 dev 0 function 0 Intel PRO/1000MT (82573E) rev 0x03: apic 2
int 16 (irq 5)PHY ID 0x1410CC0 detected (17)
, address 00:30:48:bd:45:3a
ppb4 at pci0 dev 28 function 5 Intel 82801I PCIE rev 0x02: apic 2 int 17
(irq 10)
pci5 at ppb4 bus 15
em1 at pci5 dev 0 function 0 Intel PRO/1000MT (82573L) rev 0x00: apic 2
int 17 (irq 10)PHY ID 0x1410CC0 detected (17)
, address 00:30:48:bd:45:3b
uhci3 at pci0 dev 29 function 0 Intel 82801I USB rev 0x02: apic 2 int 23
(irq 10)
uhci4 at pci0 dev 29 function 1 Intel 82801I USB rev 0x02: apic 2 int 22
(irq 11)
uhci5 at pci0 dev 29 function 2 Intel 82801I USB rev 0x02: apic 2 int 18
(irq 11)
ehci1 at pci0 dev 29 function 7 Intel 82801I USB rev 0x02: apic 2 int 23
(irq 10)

Re: unidentified system load

[Tony Sarendal]

 I'd be looking at the state of your mbufs as well.  man netstat


Thanks Aaron,

these systems are currently running with load very low. From one of the
boxes with
the problem:

bmr1.mlt# uptime
11:33AM  up 13 days,  1:04, 1 user, load averages: 0.15, 0.17, 0.11
bmr1.mlt# netstat -m
102 mbufs in use:
81 mbufs allocated to data
4 mbufs allocated to packet headers
17 mbufs allocated to socket names and addresses
69/310/6144 mbuf 2048 byte clusters in use (current/peak/max)
0/8/6144 mbuf 4096 byte clusters in use (current/peak/max)
0/8/6144 mbuf 8192 byte clusters in use (current/peak/max)
0/8/6144 mbuf 9216 byte clusters in use (current/peak/max)
0/8/6144 mbuf 12288 byte clusters in use (current/peak/max)
0/8/6144 mbuf 16384 byte clusters in use (current/peak/max)
0/8/6144 mbuf 65536 byte clusters in use (current/peak/max)
900 Kbytes allocated to network (18% in use)
0 requests for memory denied
0 requests for memory delayed
0 calls to protocol drain routines
bmr1.mlt#
bmr1.mlt# vmstat -m
...
Memory Totals:  In UseFreeRequests
 3756K210K 3844873
...
In use 6468K, total allocated 32928K; utilization 19.6%
bmr1.mlt#

They are basically standard setups with ipsec,bgpd,gre,carp and vlans,
small and simple configs, low number of interfaces. Not even any packages
added.

IPsec 4.6 to snapshot failing

[Tony Sarendal]

Good morning misc,

I upgraded two devices from i386-4.6 to i386-snapshot-feb28.
After the upgrade snapshot boxes are unable to communicate with the 4.6
devices
when going through ipsec. snapshot-snapshot works fine.

Everything looks ok except that nothing shows up on enc0 when doing
4.6--snapshot.
Deleting the SA's restores connectiviy, unencrypted of course.
Is this a known issue ?

/T

bmr1.jfa: 212.112.186.174 (4.6)
bmr1.brh: 212.188.183.71 (snapshot)

---
bmr1.jfa# ipsecctl -sa | grep 212.188.183.71
flow esp in from 212.188.183.71 to 212.112.186.174 peer 212.188.183.71 srcid
212.112.186.174/32 dstid 212.188.183.71/32 type use
flow esp out from 212.112.186.174 to 212.188.183.71 peer 212.188.183.71
srcid 212.112.186.174/32 dstid 212.188.183.71/32 type require
esp transport from 212.188.183.71 to 212.112.186.174 spi 0x3f91b3c2 auth
hmac-sha2-256 enc aes
esp transport from 212.112.186.174 to 212.188.183.71 spi 0xa797ec1e auth
hmac-sha2-256 enc aes
bmr1.jfa#

bmr1.brh# ipsecctl -sa | grep 212.112.186.174
flow esp in from 212.112.186.174 to 212.188.183.71 peer 212.112.186.174
srcid 212.188.183.71/32 dstid 212.112.186.174/32 type use
flow esp out from 212.188.183.71 to 212.112.186.174 peer 212.112.186.174
srcid 212.188.183.71/32 dstid 212.112.186.174/32 type require
esp transport from 212.188.183.71 to 212.112.186.174 spi 0x3f91b3c2 auth
hmac-sha2-256 enc aes
esp transport from 212.112.186.174 to 212.188.183.71 spi 0xa797ec1e auth
hmac-sha2-256 enc aes
bmr1.brh#

bmr1.brh# pfctl -d
pf disabled
bmr1.brh# tcpdump -n -p -i vlan301 host 212.112.186.174 
[1] 2099
bmr1.brh# tcpdump: listening on vlan301, link-type EN10MB
bmr1.brh# tcpdump -n -p -i enc0 
[2] 23922
bmr1.brh# tcpdump: listening on enc0, link-type ENC
bmr1.brh#

bmr1.jfa# tcpdump -n -p -i bge0 host 212.188.183.71 
[1] 443
bmr1.jfa# tcpdump: listening on bge0, link-type EN10MB
bmr1.jfa# tcpdump -n -p -i enc0 
[2] 16714
bmr1.jfa# tcpdump: listening on enc0, link-type ENC
bmr1.jfa#


bmr1.jfa# ping 212.188.183.71
PING 212.188.183.71 (212.188.183.71): 56 data bytes
11:21:48.081933 (authentic,confidential): SPI 0x007e7833: 212.112.186.174 
212.188.183.71: icmp: echo request
11:21:48.081969 esp 212.112.186.174  212.188.183.71 spi 0x007e7833 seq 15
len 116
11:21:49.085937 (authentic,confidential): SPI 0x007e7833: 212.112.186.174 
212.188.183.71: icmp: echo request
11:21:49.085974 esp 212.112.186.174  212.188.183.71 spi 0x007e7833 seq 16
len 116
11:21:50.095970 (authentic,confidential): SPI 0x007e7833: 212.112.186.174 
212.188.183.71: icmp: echo request
11:21:50.096006 esp 212.112.186.174  212.188.183.71 spi 0x007e7833 seq 17
len 116
11:21:51.106010 (authentic,confidential): SPI 0x007e7833: 212.112.186.174 
212.188.183.71: icmp: echo request
11:21:51.106045 esp 212.112.186.174  212.188.183.71 spi 0x007e7833 seq 18
len 116

bmr1.brh# 10:21:48.102134 esp 212.112.186.174  212.188.183.71 spi
0x007e7833 seq 15 len 116
10:21:49.106079 esp 212.112.186.174  212.188.183.71 spi 0x007e7833 seq 16
len 116
10:21:50.116146 esp 212.112.186.174  212.188.183.71 spi 0x007e7833 seq 17
len 116
10:21:51.126213 esp 212.112.186.174  212.188.183.71 spi 0x007e7833 seq 18
len 116



bmr1.jfa# grep 212.188.183.71 /etc/ipsec.conf
ike esp transport from 212.112.186.174 to 212.188.183.71

bmr1.brh# grep 212.112.186.174 /etc/ipsec.conf
ike esp transport from 212.188.183.71 to 212.112.186.174

Re: IPsec 4.6 to snapshot failing

[Tony Sarendal]

On Mon, Mar 1, 2010 at 12:54 PM, Stuart Henderson s...@spacehopper.orgwrote:

 On 2010-03-01, Tony Sarendal t...@polarcap.org wrote:
  Good morning misc,
 
  I upgraded two devices from i386-4.6 to i386-snapshot-feb28.
  After the upgrade snapshot boxes are unable to communicate with the 4.6
  devices
  when going through ipsec. snapshot-snapshot works fine.
 
  Everything looks ok except that nothing shows up on enc0 when doing
  4.6--snapshot.
  Deleting the SA's restores connectiviy, unencrypted of course.
  Is this a known issue ?

 yes, there was a bug with hmac-sha2 which was causing interop problems
 with correct IPsec implementations and needed fixing, unfortunately the
 fix breaks backwards compatibility.

 you'll need to switch to e.g. hmac-sha until the 4.6 box can be upgraded.


Thanks to everyone for the quick and correct response, much appreciated.

/T, with a tad of ring rust.

Re: Routing between spokes - recent best practices?

[Tony Sarendal]

On 12/4/07, John Rodenbiker [EMAIL PROTECTED] wrote:

 On Dec 4, 2007, at 12:14 AM, visc wrote:
  So, my question is this - what are the current best practices for
  setting up a hub and spoke topology using OpenBSD, allowing for
  traffic to securely flow from Branch to Branch on occasion without
  using a full mesh topology. If it's at all possible... (network
  description below)

 At this point IMHO branch-to-branch is avoided not for security
 reasons but for administrative reasons.

 It is a pain in the ass to configure each branch to establish a VPN to
 any other branch. It's easy to tell each branch router if you want to
 talk to BRANCHX, talk to CENTRALOFFICE first.


GRE/IPIP inside IPsec and dynamic routing.

/Tony


If you have more than a handful of branches it is very annoying to
 tell each router if you want to talk to BRACHA, talk to A; if you
 want to talk to BRANCHB, talk to B; etc.

 The primary advantage of the star or branch-to-central topology was
 the difficulty of someone putting a man-in-the-middle of a leased line.

 But now leased lines are expensive. VPNs and direct Internet
 connections are cheap so it makes much more sense to put in the pain-
 in-the-ass effort to connect everyone in your Intranet via VPNs/IPSEC
 and get rid of your leased lines.

 If you only have enough budget to move a few this year you analyze
 which few cross-talk the most and configure them for mesh and leave
 the rest as star.

 This is not true if you asked an auditor, however. It is much easier
 to put a network sensor down in a star topology and get most of the
 network traffic than it is for a mesh network. If you want to be able
 to buy one device and know for sure that everyone is going through it
 you probably need a star topology and a heavy hand on the branch
 routers.
 --
 Freedom, truth, love, beauty.
 John Rodenbiker

Re: Routing between spokes - recent best practices?

[Tony Sarendal]

On 12/4/07, Tony Sarendal [EMAIL PROTECTED] wrote:



 On 12/4/07, John Rodenbiker [EMAIL PROTECTED] wrote:
 
  On Dec 4, 2007, at 12:14 AM, visc wrote:
   So, my question is this - what are the current best practices for
   setting up a hub and spoke topology using OpenBSD, allowing for
   traffic to securely flow from Branch to Branch on occasion without
   using a full mesh topology. If it's at all possible... (network
   description below)
 
  At this point IMHO branch-to-branch is avoided not for security
  reasons but for administrative reasons.
 
  It is a pain in the ass to configure each branch to establish a VPN to
  any other branch. It's easy to tell each branch router if you want to
  talk to BRANCHX, talk to CENTRALOFFICE first.


 GRE/IPIP inside IPsec and dynamic routing.


Or just a management tool to create configs and push it out.

/Tony


/Again

Re: bgpd patch, WAS: bgpd causing black-holes with bgp-only setup

[Tony Sarendal]

On 11/12/07, Claudio Jeker [EMAIL PROTECTED] wrote:

 On Tue, Nov 06, 2007 at 06:26:47PM +0100, Tony Sarendal wrote:
  New version. Less duplication and a nice feature as bonus.
  With softreconfig in enabled the looped prefixes are accepted
  into the Adj-RIB-In.
 
  This means that I can tell if my neighbor AS is using
  a path via myself. Either I'm tired or that is cool.
 
  router-02# bgpctl show rib 192.168.0.0
  flags: * = Valid,  = Selected, I = via IBGP, A = Announced
  origin: i = IGP, e = EGP, ? = Incomplete
 
  flags destination gateway  lpref   med aspath origin
  *192.168.0.0/16  192.168.100.5  100 0 65100 i
  * 192.168.0.0/16  172.17.1.1 100 0 65200 65100 i
  * 192.168.0.0/16  172.17.1.5 100 0 65200 65200 65200
 65200 65100 i
  router-02#
 
  I now kill the peering that 65200 has to 65100, removing their
  direct path to 192.168.0.0/16.
 
  router-02# bgpctl show rib 192.168.0.0
  flags: * = Valid,  = Selected, I = via IBGP, A = Announced
  origin: i = IGP, e = EGP, ? = Incomplete
 
  flags destination gateway  lpref   med aspath origin
  *192.168.0.0/16  192.168.100.5  100 0 65100 i
  router-02#
 
  Sweet, the looping issue is gone.
  Here is the bonus:
 
  router-02# bgpctl show rib neigh 172.17.1.5 in  | grep 65300
  * 172.17.0.2/32   172.17.1.5 100 0 65200 65300 i
  * 192.168.0.0/16  172.17.1.5 100 0 65200 65300 65100
 i
  * 192.168.100.4/30172.17.1.5 100 0 65200 65300 i
  router-02#
 
  I now see the paths that the peer uses my network to access.
  Note that this depends a bit on remote implementation.
  I think this works agains a cisco router.
 
  /Tony
 
 
  Index: rde.c
  ===
  RCS file: /cvs/src/usr.sbin/bgpd/rde.c,v
  retrieving revision 1.228
  diff -u -r1.228 rde.c
  --- rde.c 16 Sep 2007 15:20:50 -  1.228
  +++ rde.c 6 Nov 2007 17:08:50 -
  @@ -919,12 +919,6 @@
/* shift to NLRI information */
p += 2 + attrpath_len;
 
  - /* aspath needs to be loop free nota bene this is not a hard error
 */
  - if (peer-conf.ebgp  !aspath_loopfree(asp-aspath, conf-as)) {
  - error = 0;
  - goto done;
  - }
  -
/* parse nlri prefix */
while (nlri_len  0) {
if ((pos = rde_update_get_prefix(p, nlri_len, prefix,
  @@ -977,10 +971,18 @@
if (fasp == NULL)
fasp = asp;
 
  - rde_update_log(update, peer,
 fasp-nexthop-exit_nexthop,
  - prefix, prefixlen);
  - path_update(peer, fasp, prefix, prefixlen, F_LOCAL);
  -
  + rde_update_log(update, peer,
  + fasp-nexthop-exit_nexthop,prefix,
  + prefixlen);
  + /* handle an update with loop as a withdraw */
  + if (peer-conf.ebgp  !aspath_loopfree(asp-aspath,
  + conf-as))
  + prefix_remove(peer, prefix, prefixlen,
  + F_LOCAL);
  + else
  + path_update(peer, fasp, prefix, prefixlen,
  + F_LOCAL);
  +
/* free modified aspath */
if (fasp != asp)
path_put(fasp);
  @@ -1075,9 +1077,15 @@
 
rde_update_log(update, peer,
asp-nexthop-exit_nexthop,
  - prefix, prefixlen);
  - path_update(peer, fasp, prefix,
 prefixlen,
  - F_LOCAL);
  + prefix, prefixlen);
  + /* handle an update with loop as a
 withdraw */
  + if (peer-conf.ebgp 
  +
 !aspath_loopfree(asp-aspath,conf-as))
  + prefix_remove(peer, prefix,
  + prefixlen,F_LOCAL);
  + else
  + path_update(peer, fasp, prefix,
  + prefixlen,F_LOCAL);
 
/* free modified aspath */
if (fasp != asp)

 I looked a bit closer at this problem and the RFC mentions that pathes
 with loops need to be inserted into the RIB and will be ignored in phase 2
 of the decision process.

 So this diff does just about that. It does not remove any prefix if there
 is a loop but instead is ignoring them during the route decision process.
 This seems to work for me but I'm currently unable to do larger tests.

 --
 :wq Claudio

 Index: rde.c
 ===
 RCS file: /cvs/src/usr.sbin/bgpd/rde.c,v
 retrieving revision 1.228
 diff -u -p -r1.228 rde.c
 --- rde.c   16

Re: Mysterious transfer speed differences

[Tony Sarendal]

On 11/7/07, Martin Toft [EMAIL PROTECTED] wrote:

 Hi,

 I'm experiencing some mysterious transfer speed differences. I have a
 virtual Linux-server at HostEurope, Germany, and it appears that
 machines running OpenBSD can only download from the Linux-server with
 approx 300 kB/s, whereas machines running Linux can download with approx
 1.5 MB/s from the server.

 I have conducted a small test to clarify what I'm talking about. The
 involved machines:

   - sauerkraut.obsd.dk (87.230.22.203) (Debian GNU/Linux, 2.6.9 kernel).
   - gw.obelnet.dk (130.225.243.84) (OpenBSD 4.2-stable).
   - matrix.math.aau.dk (130.225.48.12) (Ubuntu Linux, 2.6.20 kernel).

 sauerkraut.obsd.dk is the server at HostEurope, and it offers a 50 MB
 download
 at http://obsd.dk/50MB, which I'll download from the other two machines.

 gw.obelnet.dk:

 [EMAIL PROTECTED]:~$ wget -O /dev/null http://obsd.dk/50MB
 --21:03:22--  http://obsd.dk/50MB
= `/dev/null'
 Resolving obsd.dk... 87.230.22.203
 Connecting to obsd.dk|87.230.22.203|:80... connected.
 HTTP request sent, awaiting response... 200 OK
 Length: 52,428,800 (50M) [text/plain]

 100%[] 52,428,800   334.12K/sETA
 00:00

 21:05:57 (331.55 KB/s) - `/dev/null' saved [52428800/52428800]

 matrix.math.aau.dk:

 [EMAIL PROTECTED]:~$ wget -O /dev/null http://obsd.dk/50MB
 --21:06:45--  http://obsd.dk/50MB
= `/dev/null'
 Resolving obsd.dk... 87.230.22.203
 Connecting to obsd.dk|87.230.22.203|:80... connected.
 HTTP request sent, awaiting response... 200 OK
 Length: 52.428.800 (50M) [text/plain]

 100%[] 52.428.800 1.59M/sETA
 00:00

 21:07:17 (1.56 MB/s) - `/dev/null' saved [52428800/52428800]

 I have attached three traceroutes -- from gw.obelnet.dk to
 sauerkraut.obsd.dk,
 from matrix.math.aau.dk to sauerkraut.obsd.dk, and from gw.obelnet.dk to
 matrix.math.aau.dk. The last one illustrates how close gw.obelnet.dk and
 matrix.math.aau.dk are located, network wise. Both are connected through
 Aalborg University, which is connected through the Danish Network for
 Research
 and Education, which again is connected through NORDUnet (a collaboration
 between nordic research networks).

 What baffles me most is that when I make a ssh tunnel from gw.obelnet.dk
 through matrix.math.aau.dk to sauerkraut.obsd.dk, I can download the 50
 MB file with approx 1.5 MB/s:

 [EMAIL PROTECTED]:~$ grep obsd.dk /etc/hosts
 127.0.0.1 localhost.obelnet.dk localhost obsd.dk
 [EMAIL PROTECTED]:~$ ssh -N -L 4000:obsd.dk:80 matrix.math.aau.dk 
 [EMAIL PROTECTED]:~$ wget -O /dev/null http://obsd.dk:4000/50MB
 --21:26:23--  http://obsd.dk:4000/50MB
= `/dev/null'
 Resolving obsd.dk... 127.0.0.1
 Connecting to obsd.dk|127.0.0.1|:4000... connected.
 HTTP request sent, awaiting response... 200 OK
 Length: 52,428,800 (50M) [text/plain]

 100%[] 52,428,800 1.47M/sETA
 00:00

 21:26:55 (1.54 MB/s) - `/dev/null' saved [52428800/52428800]

 Beside the test above, I have also downloaded the 50 MB file from a
 Linux box (Debian GNU/Linux, 2.6.x kernel) and an OpenBSD box
 (4.2-current), both directly behind gw.obelnet.dk. gw.obelnet.dk is
 a NAT gateway for the two boxes, as you might have guessed. The numbers
 from above are confirmed -- the Linux box downloads with 1-1.5 MB/s, and
 the OpenBSD box downloads with 270-300 kB/s.

 Can anybody explain these differences? Why are the downloads so slow on
 OpenBSD?

 Any help is appreciated. If you think I've left something important out,
 don't hesitate to ask.

 Best regards,
 Martin


 Attachments:

 Traceroute from gw.obelnet.dk to sauerkraut.obsd.dk:

 [EMAIL PROTECTED]:~$ traceroute -P 1 sauerkraut.obsd.dk
 traceroute to sauerkraut.obsd.dk (87.230.22.203), 64 hops max, 60 byte
 packets
  1  gi1-0-1.aalborg1.aau.dk (130.225.243.65)  0.769 ms  0.814 ms
 0.802 ms
  2  icdata.ly4.core.fsknet.dk (130.225.242.34)  6.758 ms  6.392 ms
 6.606 ms
  3  teng-ly4.ly3.core.fsknet.dk (130.225.244.145)  6.275 ms  6.634ms
 6.683 ms
  4  10g-ly3.or1.core.fsknet.dk (130.225.244.218)  6.615 ms  7.184 ms
 6.924 ms
  5  dk-ore.nordu.net (193.10.68.121)  6.791 ms  6.726 ms  6.608 ms
  6  se-fre.nordu.net (193.10.68.117)  16.359 ms  16.418 ms  16.604 ms
  7  s-b3-link.telia.net (213.248.97.17)  20.683 ms  20.695 ms  17.285ms
  8  s-bb1-link.telia.net (80.91.254.58)  20.385 ms  20.360 ms  16.842ms
  9  ffm-bb1-link.telia.net (80.91.251.145)  43.569 ms  44.25 ms
 47.294 ms
 10  ffm-b1-link.telia.net (80.91.254.97)  47.347 ms  47.275 ms  47.236ms
 11  pipex-115772-ffm-b1.c.telia.net (213.248.102.158)  51.53 ms
 54.802 ms  50.290 ms
 12  xe-6-1-0.juwel.cgn3.hosteurope.de (80.237.129.114)  49.954 ms
 49.954 ms  50.155 ms
 13  xe-16-1.mlx31m.cgn3.hosteurope.de 

Re: bgpd patch, WAS: bgpd causing black-holes with bgp-only setup

[Tony Sarendal]

diff -u version.

/Tony


Index: rde.c
===
RCS file: /cvs/src/usr.sbin/bgpd/rde.c,v
retrieving revision 1.228
diff -u -r1.228 rde.c
--- rde.c   16 Sep 2007 15:20:50 -  1.228
+++ rde.c   6 Nov 2007 10:38:23 -
@@ -919,12 +919,6 @@
/* shift to NLRI information */
p += 2 + attrpath_len;
 
-   /* aspath needs to be loop free nota bene this is not a hard error */
-   if (peer-conf.ebgp  !aspath_loopfree(asp-aspath, conf-as)) {
-   error = 0;
-   goto done;
-   }
-
/* parse nlri prefix */
while (nlri_len  0) {
if ((pos = rde_update_get_prefix(p, nlri_len, prefix,
@@ -954,9 +948,17 @@
 
peer-prefix_rcvd_update++;
/* add original path to the Adj-RIB-In */
-   if (peer-conf.softreconfig_in)
-   path_update(peer, asp, prefix, prefixlen, F_ORIGINAL);
-
+   if (peer-conf.softreconfig_in) {
+   /* handle an update with loop as a withdraw */
+   if (peer-conf.ebgp  !aspath_loopfree(asp-aspath,
+   conf-as))
+   prefix_remove(peer, prefix, prefixlen,
+   F_ORIGINAL);
+   else
+   path_update(peer, asp, prefix, prefixlen,
+   F_ORIGINAL);
+   
+   }
/* input filter */
if (rde_filter(fasp, rules_l, peer, asp, prefix, prefixlen,
peer, DIR_IN) == ACTION_DENY) {
@@ -977,10 +979,18 @@
if (fasp == NULL)
fasp = asp;
 
-   rde_update_log(update, peer, fasp-nexthop-exit_nexthop,
-   prefix, prefixlen);
-   path_update(peer, fasp, prefix, prefixlen, F_LOCAL);
-
+   rde_update_log(update, peer,
+   fasp-nexthop-exit_nexthop,prefix,
+   prefixlen);
+   /* handle an update with loop as a withdraw */
+   if (peer-conf.ebgp  !aspath_loopfree(asp-aspath,
+   conf-as))
+   prefix_remove(peer, prefix, prefixlen,
+   F_LOCAL);
+   else 
+   path_update(peer, fasp, prefix, prefixlen,
+   F_LOCAL);
+   
/* free modified aspath */
if (fasp != asp)
path_put(fasp);
@@ -1047,10 +1057,16 @@
 
peer-prefix_rcvd_update++;
/* add original path to the Adj-RIB-In */
-   if (peer-conf.softreconfig_in)
-   path_update(peer, asp, prefix,
-   prefixlen, F_ORIGINAL);
-
+   if (peer-conf.softreconfig_in) {
+   /* handle an update with loop as a withdraw */
+   if (peer-conf.ebgp 
+   
!aspath_loopfree(asp-aspath,conf-as))
+   prefix_remove(peer, prefix,
+   prefixlen,F_ORIGINAL);
+   else
+   path_update(peer, asp, prefix,
+   prefixlen, F_ORIGINAL);
+   }
/* input filter */
if (rde_filter(fasp, rules_l, peer, asp,
prefix, prefixlen, peer, DIR_IN) ==
@@ -1075,9 +1091,15 @@
 
rde_update_log(update, peer,
asp-nexthop-exit_nexthop,
-   prefix, prefixlen);
-   path_update(peer, fasp, prefix, prefixlen,
-   F_LOCAL);
+   prefix, prefixlen);
+   /* handle an update with loop as a withdraw */
+   if (peer-conf.ebgp 
+   !aspath_loopfree(asp-aspath,conf-as))
+   prefix_remove(peer, prefix,
+   prefixlen,F_LOCAL);
+   else 
+   path_update(peer, fasp, prefix,
+   prefixlen,F_LOCAL);
 
/* free modified aspath */
if (fasp != asp)

Re: bgpd patch, WAS: bgpd causing black-holes with bgp-only setup

[Tony Sarendal]

New version. Less duplication and a nice feature as bonus.
With softreconfig in enabled the looped prefixes are accepted
into the Adj-RIB-In.

This means that I can tell if my neighbor AS is using
a path via myself. Either I'm tired or that is cool.

router-02# bgpctl show rib 192.168.0.0 
flags: * = Valid,  = Selected, I = via IBGP, A = Announced
origin: i = IGP, e = EGP, ? = Incomplete

flags destination gateway  lpref   med aspath origin
*192.168.0.0/16  192.168.100.5  100 0 65100 i
* 192.168.0.0/16  172.17.1.1 100 0 65200 65100 i
* 192.168.0.0/16  172.17.1.5 100 0 65200 65200 65200 65200 
65100 i
router-02# 

I now kill the peering that 65200 has to 65100, removing their
direct path to 192.168.0.0/16.

router-02# bgpctl show rib 192.168.0.0 
flags: * = Valid,  = Selected, I = via IBGP, A = Announced
origin: i = IGP, e = EGP, ? = Incomplete

flags destination gateway  lpref   med aspath origin
*192.168.0.0/16  192.168.100.5  100 0 65100 i
router-02# 

Sweet, the looping issue is gone.
Here is the bonus:

router-02# bgpctl show rib neigh 172.17.1.5 in  | grep 65300
* 172.17.0.2/32   172.17.1.5 100 0 65200 65300 i
* 192.168.0.0/16  172.17.1.5 100 0 65200 65300 65100 i
* 192.168.100.4/30172.17.1.5 100 0 65200 65300 i
router-02# 

I now see the paths that the peer uses my network to access.
Note that this depends a bit on remote implementation.
I think this works agains a cisco router.

/Tony


Index: rde.c
===
RCS file: /cvs/src/usr.sbin/bgpd/rde.c,v
retrieving revision 1.228
diff -u -r1.228 rde.c
--- rde.c   16 Sep 2007 15:20:50 -  1.228
+++ rde.c   6 Nov 2007 17:08:50 -
@@ -919,12 +919,6 @@
/* shift to NLRI information */
p += 2 + attrpath_len;
 
-   /* aspath needs to be loop free nota bene this is not a hard error */
-   if (peer-conf.ebgp  !aspath_loopfree(asp-aspath, conf-as)) {
-   error = 0;
-   goto done;
-   }
-
/* parse nlri prefix */
while (nlri_len  0) {
if ((pos = rde_update_get_prefix(p, nlri_len, prefix,
@@ -977,10 +971,18 @@
if (fasp == NULL)
fasp = asp;
 
-   rde_update_log(update, peer, fasp-nexthop-exit_nexthop,
-   prefix, prefixlen);
-   path_update(peer, fasp, prefix, prefixlen, F_LOCAL);
-
+   rde_update_log(update, peer,
+   fasp-nexthop-exit_nexthop,prefix,
+   prefixlen);
+   /* handle an update with loop as a withdraw */
+   if (peer-conf.ebgp  !aspath_loopfree(asp-aspath,
+   conf-as))
+   prefix_remove(peer, prefix, prefixlen,
+   F_LOCAL);
+   else 
+   path_update(peer, fasp, prefix, prefixlen,
+   F_LOCAL);
+   
/* free modified aspath */
if (fasp != asp)
path_put(fasp);
@@ -1075,9 +1077,15 @@
 
rde_update_log(update, peer,
asp-nexthop-exit_nexthop,
-   prefix, prefixlen);
-   path_update(peer, fasp, prefix, prefixlen,
-   F_LOCAL);
+   prefix, prefixlen);
+   /* handle an update with loop as a withdraw */
+   if (peer-conf.ebgp 
+   !aspath_loopfree(asp-aspath,conf-as))
+   prefix_remove(peer, prefix,
+   prefixlen,F_LOCAL);
+   else 
+   path_update(peer, fasp, prefix,
+   prefixlen,F_LOCAL);
 
/* free modified aspath */
if (fasp != asp)
-- 
---
Tony Sarendal - [EMAIL PROTECTED]
IP/Unix
-= The scorpion replied,
I couldn't help it, it's my nature =-

bgpd patch, WAS: bgpd causing black-holes with bgp-only setup

[Tony Sarendal]

I have not yet checked how other implementations handle the
situation where an update with a as-path loop hides the fact
that the neighbor just lost a path.

But I made a quick patch if anyone feel like testing.
The black-hole condition does not appear anymore when
I test.

Be gentle, I only browsed through the code while on
the underground to and from work.

/Tony

Index: rde.c
===
RCS file: /cvs/src/usr.sbin/bgpd/rde.c,v
retrieving revision 1.228
diff -r1.228 rde.c
922,927d921
   /* aspath needs to be loop free nota bene this is not a hard error */
   if (peer-conf.ebgp  !aspath_loopfree(asp-aspath, conf-as)) {
   error = 0;
   goto done;
   }
 
957,959c951,961
   if (peer-conf.softreconfig_in)
   path_update(peer, asp, prefix, prefixlen, F_ORIGINAL);
 
---
   if (peer-conf.softreconfig_in) {
   /* handle an update with loop as a withdraw */
   if (peer-conf.ebgp  !aspath_loopfree(asp-aspath,
   conf-as))
   prefix_remove(peer, prefix, prefixlen,
   F_ORIGINAL);
   else
   path_update(peer, asp, prefix, prefixlen,
   F_ORIGINAL);
   
   }
980,983c982,993
   rde_update_log(update, peer, fasp-nexthop-exit_nexthop,
   prefix, prefixlen);
   path_update(peer, fasp, prefix, prefixlen, F_LOCAL);
 
---
   rde_update_log(update, peer,
   fasp-nexthop-exit_nexthop,prefix,
   prefixlen);
   /* handle an update with loop as a withdraw */
   if (peer-conf.ebgp  !aspath_loopfree(asp-aspath,
   conf-as))
   prefix_remove(peer, prefix, prefixlen,
   F_LOCAL);
   else 
   path_update(peer, fasp, prefix, prefixlen,
   F_LOCAL);
   
1050,1053c1060,1069
   if (peer-conf.softreconfig_in)
   path_update(peer, asp, prefix,
   prefixlen, F_ORIGINAL);
 
---
   if (peer-conf.softreconfig_in) {
   /* handle an update with loop as a withdraw */
   if (peer-conf.ebgp 
   
 !aspath_loopfree(asp-aspath,conf-as))
   prefix_remove(peer, prefix,
   prefixlen,F_ORIGINAL);
   else
   path_update(peer, asp, prefix,
   prefixlen, F_ORIGINAL);
   }
1078,1080c1094,1102
   prefix, prefixlen);
   path_update(peer, fasp, prefix, prefixlen,
   F_LOCAL);
---
   prefix, prefixlen);
   /* handle an update with loop as a withdraw */
   if (peer-conf.ebgp 
   !aspath_loopfree(asp-aspath,conf-as))
   prefix_remove(peer, prefix,
   prefixlen,F_LOCAL);
   else 
   path_update(peer, fasp, prefix,
   prefixlen,F_LOCAL);

bgpd causing black-holes with bgp-only setup

[Tony Sarendal]

bgpd does not re-route correctly when I shut down a transit when I
use a bgp-only design, causing black-holes for some prefixes.

router-01 and router-02 are in the same AS and peer with the same transit
provider.
router-01 and router-02 have two ibgp peerings, primary and standby path.
router-01 sets localpref 60 on all transit prefixes, router-02 sets
local-pref 50.
When I take down the transit on router-01 I see this on router-02:

router-02# bgpctl show rib | head -n 10
flags: * = Valid,  = Selected, I = via IBGP, A = Announced
origin: i = IGP, e = EGP, ? = Incomplete

flags destination gateway  lpref   med aspath origin
I*   26.0.128.0/17   172.17.1.1  60 11100 65100 i
* 26.0.128.0/17   192.168.100.5   50 10100 65100 i
I*   26.0.144.0/22   172.17.1.1  60 11100 65100 i
* 26.0.144.0/22   192.168.100.5   50 10100 65100 i
I*   26.1.77.0/24172.17.1.1  60 11100 65100 i
* 26.1.77.0/24192.168.100.5   50 10100 65100 i
router-02#

prefixes with local-pref 60 pointing at router-01.
router-01 does not have it's transit peering up, and thus itself has no
prefixes with local-pref 60.

router-01# bgpctl show rib | head -n
10

flags: * = Valid,  = Selected, I = via IBGP, A = Announced
origin: i = IGP, e = EGP, ? = Incomplete

flags destination gateway  lpref   med aspath origin
I*   26.0.128.0/17   172.17.1.6  50 21100 65100 i
I*   26.0.144.0/22   172.17.1.6  50 21100 65100 i
I*   26.1.77.0/24172.17.1.6  50 21100 65100 i
I*   26.2.172.0/22   172.17.1.6  50 21100 65100 i
I*   26.3.241.0/24   172.17.1.6  50 21100 65100 i
I*   26.6.126.0/24   172.17.1.6  50 21100 65100 i
router-01#  bgpctl show rib 26.0.128.0/17 all
flags: * = Valid,  = Selected, I = via IBGP, A = Announced
origin: i = IGP, e = EGP, ? = Incomplete

flags destination gateway  lpref   med aspath origin
I*   26.0.128.0/17   172.17.1.6  50 21100 65100 i
I*   26.0.144.0/22   172.17.1.6  50 21100 65100 i
router-01#

I saw this before when I tested bgpd around a year ago. So it isn't a new
bug.
This is with 4.2-RELEASE, no patches.

This info is from a lab I setup to replicate a live environment.


/Tony


router-01# cat /etc/bgpd.conf
# $OpenBSD: bgpd.conf,v 1.8 2007/03/29 13:37:35 claudio Exp $
# sample bgpd configuration file
# see bgpd.conf(5)

#macros
loopback=172.17.0.1

# global configuration
AS 65200
router-id $loopback

network $loopback/32 set {localpref 120, med 10}
network 172.17.0.0/16 set {localpref 120, med 10}
network connected set {localpref 120, med 10}
network static set {localpref 120, med 10}

group TRANSIT {
remote-as 65100
announce all
set nexthop self
set med 10100
set localpref 60
neighbor 192.168.100.1 {
descr TRANSIT
}
}

group IBGP {
remote-as 65200
route-reflector
set nexthop self
set med +1000
neighbor 172.17.1.2 {
local-address 172.17.1.1
descr router-02 primary
}
neighbor 172.17.1.6 {
local-address 172.17.1.5
descr router-02 standby
set med +1
}
}


# filter
deny from any
deny to any

allow quick to group IBGP
allow quick from group IBGP

allow quick to group TRANSIT prefix 172.17.0.0/16
allow quick from group TRANSIT

router-01#
ifconfig

lo0: flags=8049UP,LOOPBACK,RUNNING,MULTICAST mtu 33208
groups: lo
inet 127.0.0.1 netmask 0xff00
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
ne3: flags=8863UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST mtu 1500
lladdr 52:54:00:12:02:01
description: transit
media: Ethernet 10baseT full-duplex
inet6 fe80::5054:ff:fe12:201%ne3 prefixlen 64 scopeid 0x1
inet 192.168.100.2 netmask 0xfffc broadcast 192.168.100.3
ne4: flags=8863UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST mtu 1500
lladdr 52:54:00:12:02:02
description: router-01 primary path
media: Ethernet 10baseT full-duplex
inet6 fe80::5054:ff:fe12:202%ne4 prefixlen 64 scopeid 0x2
inet 172.17.1.1 netmask 0xfffc broadcast 172.17.1.3
ne5: flags=8863UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST mtu 1500
lladdr 52:54:00:12:02:03
description: route-02 standby path
media: Ethernet 10baseT full-duplex
inet6 fe80::5054:ff:fe12:203%ne5 prefixlen 64 scopeid 0x3
inet 172.17.1.5 netmask 0xfffc broadcast 172.17.1.7
enc0: flags=0 mtu 1536
lo1: flags=8049UP,LOOPBACK,RUNNING,MULTICAST mtu 33208
description: ROUTING LOOPBACK
groups: lo
inet 172.17.0.1 netmask 0x
router-01#





router-02# cat /etc/bgpd.conf
# $OpenBSD: bgpd.conf,v 1.8 2007/03/29 13:37:35 claudio Exp $
# sample bgpd configuration 

Re: bgpd causing black-holes with bgp-only setup

[Tony Sarendal]

On 11/4/07, Tony Sarendal [EMAIL PROTECTED] wrote:


 bgpd does not re-route correctly when I shut down a transit when I
 use a bgp-only design, causing black-holes for some prefixes.

 router-01 and router-02 are in the same AS and peer with the same transit
 provider.
 router-01 and router-02 have two ibgp peerings, primary and standby path.
 router-01 sets localpref 60 on all transit prefixes, router-02 sets
 local-pref 50.
 When I take down the transit on router-01 I see this on router-02:

 router-02# bgpctl show rib | head -n 10
 flags: * = Valid,  = Selected, I = via IBGP, A = Announced
 origin: i = IGP, e = EGP, ? = Incomplete

 flags destination gateway  lpref   med aspath origin
 I*   26.0.128.0/17   172.17.1.1  60 11100 65100 i
 * 26.0.128.0/17   192.168.100.5   50 10100 65100 i
 I*   26.0.144.0/22   172.17.1.1  60 11100 65100 i
 * 26.0.144.0/22192.168.100.5   50 10100 65100 i
 I*   26.1.77.0/24172.17.1.1  60 11100 65100 i
 * 26.1.77.0/24192.168.100.5   50 10100 65100 i
 router-02#

 prefixes with local-pref 60 pointing at router-01.
 router-01 does not have it's transit peering up, and thus itself has no
 prefixes with local-pref 60.

 router-01# bgpctl show rib | head -n
 10

 flags: * = Valid,  = Selected, I = via IBGP, A = Announced
 origin: i = IGP, e = EGP, ? = Incomplete

 flags destination gateway  lpref   med aspath origin
 I*   26.0.128.0/17   172.17.1.6   50 21100 65100 i
 I*   26.0.144.0/22   172.17.1.6  50 21100 65100 i
 I*   26.1.77.0/24 172.17.1.6  50 21100 65100 i
 I*   26.2.172.0/22   172.17.1.6  50 21100 65100 i
 I*   26.3.241.0/24   172.17.1.6  50 21100 65100 i
 I*   26.6.126.0/24   172.17.1.6   50 21100 65100 i
 router-01#  bgpctl show rib 26.0.128.0/17 all
 flags: * = Valid,  = Selected, I = via IBGP, A = Announced
 origin: i = IGP, e = EGP, ? = Incomplete

 flags destination gateway  lpref   med aspath origin
 I*   26.0.128.0/17   172.17.1.6  50 21100 65100 i
 I*   26.0.144.0/22   172.17.1.6  50 21100 65100 i
 router-01#

 I saw this before when I tested bgpd around a year ago. So it isn't a new
 bug.
 This is with 4.2-RELEASE, no patches.

 This info is from a lab I setup to replicate a live environment.


 /Tony


 router-01# cat /etc/bgpd.conf
 # $OpenBSD: bgpd.conf,v 1.8 2007/03/29 13:37:35 claudio Exp $
 # sample bgpd configuration file
 # see bgpd.conf(5)

 #macros
 loopback=172.17.0.1

 # global configuration
 AS 65200
 router-id $loopback

 network $loopback/32 set {localpref 120, med 10}
 network 172.17.0.0/16 set {localpref 120, med 10}
 network connected set {localpref 120, med 10}
 network static set {localpref 120, med 10}

 group TRANSIT {
 remote-as 65100
 announce all
 set nexthop self
 set med 10100
 set localpref 60
 neighbor 192.168.100.1 {
 descr TRANSIT
 }
 }

 group IBGP {
 remote-as 65200
 route-reflector
 set nexthop self
 set med +1000
 neighbor 172.17.1.2 {
 local-address 172.17.1.1
 descr router-02 primary
 }
 neighbor 172.17.1.6 {
 local-address 172.17.1.5
 descr router-02 standby
 set med +1
 }
 }


 # filter
 deny from any
 deny to any

 allow quick to group IBGP
 allow quick from group IBGP

 allow quick to group TRANSIT prefix 172.17.0.0/16
 allow quick from group TRANSIT

 router-01#
 ifconfig

 lo0: flags=8049UP,LOOPBACK,RUNNING,MULTICAST mtu 33208
 groups: lo
 inet 127.0.0.1 netmask 0xff00
 inet6 ::1 prefixlen 128
 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
 ne3: flags=8863UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST mtu
 1500
 lladdr 52:54:00:12:02:01
 description: transit
 media: Ethernet 10baseT full-duplex
 inet6 fe80::5054:ff:fe12:201%ne3 prefixlen 64 scopeid 0x1
 inet 192.168.100.2 netmask 0xfffc broadcast 192.168.100.3
 ne4: flags=8863UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST mtu
 1500
 lladdr 52:54:00:12:02:02
 description: router-01 primary path
 media: Ethernet 10baseT full-duplex
 inet6 fe80::5054:ff:fe12:202%ne4 prefixlen 64 scopeid 0x2
 inet 172.17.1.1 netmask 0xfffc broadcast 172.17.1.3
 ne5: flags=8863UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST mtu
 1500
 lladdr 52:54:00:12:02:03
 description: route-02 standby path
 media: Ethernet 10baseT full-duplex
 inet6 fe80::5054:ff:fe12:203%ne5 prefixlen 64 scopeid 0x3
 inet 172.17.1.5 netmask 0xfffc broadcast 172.17.1.7
 enc0: flags=0 mtu 1536
 lo1: flags=8049UP,LOOPBACK,RUNNING,MULTICAST mtu 33208
 description: ROUTING LOOPBACK
 groups: lo

Re: bgpd causing black-holes with bgp-only setup

[Tony Sarendal]

On 11/4/07, Tony Sarendal [EMAIL PROTECTED] wrote:

 On 11/4/07, Tony Sarendal [EMAIL PROTECTED] wrote:

 
  bgpd does not re-route correctly when I shut down a transit when I
  use a bgp-only design, causing black-holes for some prefixes.
 
  router-01 and router-02 are in the same AS and peer with the same
  transit provider.
  router-01 and router-02 have two ibgp peerings, primary and standby
  path.
  router-01 sets localpref 60 on all transit prefixes, router-02 sets
  local-pref 50.
  When I take down the transit on router-01 I see this on router-02:
 
  router-02# bgpctl show rib | head -n 10
  flags: * = Valid,  = Selected, I = via IBGP, A = Announced
  origin: i = IGP, e = EGP, ? = Incomplete
 
  flags destination gateway  lpref   med aspath origin
  I*   26.0.128.0/17   172.17.1.1  60 11100 65100 i
  * 26.0.128.0/17   192.168.100.5   50 10100 65100 i
  I*   26.0.144.0/22   172.17.1.1  60 11100 65100 i
  * 26.0.144.0/22192.168.100.5   50 10100 65100 i
  I*   26.1.77.0/24172.17.1.1  60 11100 65100 i
  * 26.1.77.0/24192.168.100.5   50 10100 65100 i
  router-02#
 
  prefixes with local-pref 60 pointing at router-01.
  router-01 does not have it's transit peering up, and thus itself has no
  prefixes with local-pref 60.
 
  router-01# bgpctl show rib | head -n
  10
 
  flags: * = Valid,  = Selected, I = via IBGP, A = Announced
  origin: i = IGP, e = EGP, ? = Incomplete
 
  flags destination gateway  lpref   med aspath origin
  I*   26.0.128.0/17   172.17.1.6   50 21100 65100 i
  I*   26.0.144.0/22   172.17.1.6  50 21100 65100 i
  I*   26.1.77.0/24 172.17.1.6  50 21100 65100 i
  I*   26.2.172.0/22   172.17.1.6  50 21100 65100 i
  I*   26.3.241.0/24   172.17.1.6  50 21100 65100 i
  I*   26.6.126.0/24   172.17.1.6   50 21100 65100 i
  router-01#  bgpctl show rib 26.0.128.0/17 all
  flags: * = Valid,  = Selected, I = via IBGP, A = Announced
  origin: i = IGP, e = EGP, ? = Incomplete
 
  flags destination gateway  lpref   med aspath origin
  I*   26.0.128.0/17   172.17.1.6  50 21100 65100 i
  I*   26.0.144.0/22   172.17.1.6  50 21100 65100 i
  router-01#
 
  I saw this before when I tested bgpd around a year ago. So it isn't a
  new bug.
  This is with 4.2-RELEASE, no patches.
 
  This info is from a lab I setup to replicate a live environment.
 
 
  /Tony
 
 
  router-01# cat /etc/bgpd.conf
  # $OpenBSD: bgpd.conf,v 1.8 2007/03/29 13:37:35 claudio Exp $
  # sample bgpd configuration file
  # see bgpd.conf(5)
 
  #macros
  loopback=172.17.0.1
 
  # global configuration
  AS 65200
  router-id $loopback
 
  network $loopback/32 set {localpref 120, med 10}
  network 172.17.0.0/16 set {localpref 120, med 10}
  network connected set {localpref 120, med 10}
  network static set {localpref 120, med 10}
 
  group TRANSIT {
  remote-as 65100
  announce all
  set nexthop self
  set med 10100
  set localpref 60
  neighbor 192.168.100.1 {
  descr TRANSIT
  }
  }
 
  group IBGP {
  remote-as 65200
  route-reflector
  set nexthop self
  set med +1000
  neighbor 172.17.1.2 {
  local-address 172.17.1.1
  descr router-02 primary
  }
  neighbor 172.17.1.6 {
  local-address 172.17.1.5
  descr router-02 standby
  set med +1
  }
  }
 
 
  # filter
  deny from any
  deny to any
 
  allow quick to group IBGP
  allow quick from group IBGP
 
  allow quick to group TRANSIT prefix 172.17.0.0/16
  allow quick from group TRANSIT
 
  router-01#
  ifconfig
 
  lo0: flags=8049UP,LOOPBACK,RUNNING,MULTICAST mtu 33208
  groups: lo
  inet 127.0.0.1 netmask 0xff00
  inet6 ::1 prefixlen 128
  inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
  ne3: flags=8863UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST mtu
  1500
  lladdr 52:54:00:12:02:01
  description: transit
  media: Ethernet 10baseT full-duplex
  inet6 fe80::5054:ff:fe12:201%ne3 prefixlen 64 scopeid 0x1
  inet 192.168.100.2 netmask 0xfffc broadcast 192.168.100.3
  ne4: flags=8863UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST mtu
  1500
  lladdr 52:54:00:12:02:02
  description: router-01 primary path
  media: Ethernet 10baseT full-duplex
  inet6 fe80::5054:ff:fe12:202%ne4 prefixlen 64 scopeid 0x2
  inet 172.17.1.1 netmask 0xfffc broadcast 172.17.1.3
  ne5: flags=8863UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST mtu
  1500
  lladdr 52:54:00:12:02:03
  description: route-02 standby path
  media: Ethernet 10baseT full-duplex
  inet6 fe80::5054:ff:fe12:203%ne5 prefixlen 64 scopeid 0x3

Re: bgpd causing black-holes with bgp-only setup

[Tony Sarendal]

On 11/4/07, Tony Sarendal [EMAIL PROTECTED] wrote:



 On 11/4/07, Tony Sarendal [EMAIL PROTECTED] wrote:
 
  On 11/4/07, Tony Sarendal [EMAIL PROTECTED]  wrote:
 
  
   bgpd does not re-route correctly when I shut down a transit when I
   use a bgp-only design, causing black-holes for some prefixes.
  
   router-01 and router-02 are in the same AS and peer with the same
   transit provider.
   router-01 and router-02 have two ibgp peerings, primary and standby
   path.
   router-01 sets localpref 60 on all transit prefixes, router-02 sets
   local-pref 50.
   When I take down the transit on router-01 I see this on router-02:
  
   router-02# bgpctl show rib | head -n 10
   flags: * = Valid,  = Selected, I = via IBGP, A = Announced
   origin: i = IGP, e = EGP, ? = Incomplete
  
   flags destination gateway  lpref   med aspath origin
   I*   26.0.128.0/17   172.17.1.1  60 11100 65100 i
   * 26.0.128.0/17   192.168.100.5   50 10100 65100 i
   I*   26.0.144.0/22   172.17.1.1  60 11100 65100 i
   * 26.0.144.0/22192.168.100.5   50 10100 65100 i
   I*   26.1.77.0/24172.17.1.1  60 11100 65100 i
   * 26.1.77.0/24192.168.100.5   50 10100 65100 i
   router-02#
  
   prefixes with local-pref 60 pointing at router-01.
   router-01 does not have it's transit peering up, and thus itself has
   no prefixes with local-pref 60.
  
   router-01# bgpctl show rib | head -n
   10
  
   flags: * = Valid,  = Selected, I = via IBGP, A = Announced
   origin: i = IGP, e = EGP, ? = Incomplete
  
   flags destination gateway  lpref   med aspath origin
   I*   26.0.128.0/17   172.17.1.6   50 21100 65100 i
   I*   26.0.144.0/22   172.17.1.6  50 21100 65100 i
   I*   26.1.77.0/24 172.17.1.6  50 21100 65100 i
   I*   26.2.172.0/22   172.17.1.6  50 21100 65100 i
   I*   26.3.241.0/24   172.17.1.6  50 21100 65100 i
   I*   26.6.126.0/24   172.17.1.6   50 21100 65100 i
   router-01#  bgpctl show rib 26.0.128.0/17 all
   flags: * = Valid,  = Selected, I = via IBGP, A = Announced
   origin: i = IGP, e = EGP, ? = Incomplete
  
   flags destination gateway  lpref   med aspath origin
   I*   26.0.128.0/17   172.17.1.6  50 21100 65100 i
   I*   26.0.144.0/22   172.17.1.6  50 21100 65100 i
   router-01#
  
   I saw this before when I tested bgpd around a year ago. So it isn't a
   new bug.
   This is with 4.2-RELEASE, no patches.
  
   This info is from a lab I setup to replicate a live environment.
  
  
   /Tony
  
  
   router-01# cat /etc/bgpd.conf
   # $OpenBSD: bgpd.conf,v 1.8 2007/03/29 13:37:35 claudio Exp $
   # sample bgpd configuration file
   # see bgpd.conf(5)
  
   #macros
   loopback=172.17.0.1
  
   # global configuration
   AS 65200
   router-id $loopback
  
   network $loopback/32 set {localpref 120, med 10}
   network 172.17.0.0/16 set {localpref 120, med 10}
   network connected set {localpref 120, med 10}
   network static set {localpref 120, med 10}
  
   group TRANSIT {
   remote-as 65100
   announce all
   set nexthop self
   set med 10100
   set localpref 60
   neighbor 192.168.100.1 {
   descr TRANSIT
   }
   }
  
   group IBGP {
   remote-as 65200
   route-reflector
   set nexthop self
   set med +1000
   neighbor 172.17.1.2 {
   local-address 172.17.1.1
   descr router-02 primary
   }
   neighbor 172.17.1.6 {
   local-address 172.17.1.5
   descr router-02 standby
   set med +1
   }
   }
  
  
   # filter
   deny from any
   deny to any
  
   allow quick to group IBGP
   allow quick from group IBGP
  
   allow quick to group TRANSIT prefix 172.17.0.0/16
   allow quick from group TRANSIT
  
   router-01#
   ifconfig
  
   lo0: flags=8049UP,LOOPBACK,RUNNING,MULTICAST mtu 33208
   groups: lo
   inet 127.0.0.1 netmask 0xff00
   inet6 ::1 prefixlen 128
   inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
   ne3: flags=8863UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST mtu
   1500
   lladdr 52:54:00:12:02:01
   description: transit
   media: Ethernet 10baseT full-duplex
   inet6 fe80::5054:ff:fe12:201%ne3 prefixlen 64 scopeid 0x1
   inet 192.168.100.2 netmask 0xfffc broadcast 192.168.100.3
   ne4: flags=8863UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST mtu
   1500
   lladdr 52:54:00:12:02:02
   description: router-01 primary path
   media: Ethernet 10baseT full-duplex
   inet6 fe80::5054:ff:fe12:202%ne4 prefixlen 64 scopeid 0x2
   inet 172.17.1.1 netmask 0xfffc broadcast 172.17.1.3
   ne5: flags=8863UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST mtu
   1500

Re: bgpd causing black-holes with bgp-only setup

[Tony Sarendal]

On 11/5/07, Claudio Jeker [EMAIL PROTECTED] wrote:

 On Sun, Nov 04, 2007 at 11:30:20PM +, Tony Sarendal wrote:
  On 11/4/07, Tony Sarendal [EMAIL PROTECTED] wrote:
  

 Thanks for all the info. I will have a look at this as well. Currently I
 think it is possible that route-reflector is not bug free in cases where
 you have route-reflector rings or other very complex setups. I only tested
 the easy setups till now. Why you get routing loops and black-holes in
 your 3 AS setups is not clear (at least for me) but I guess it may be an
 issue with a failed update. I have the feeling that when we get a update
 with a routing loop in it we should actually issue a withdraw for the
 prefix carried in it so the following code in rde.c is looking suspicious:
 /* aspath needs to be loop free nota bene this is not a hard error
 */
 if (peer-conf.ebgp  !aspath_loopfree(asp-aspath, conf-as)) {
 error = 0;
 goto done;
 }

 I'm mostly offline in the next days so maybe you beat me in finding a fix
 for this.
 --



RFC4271:

   Changing the attribute(s) of a route is accomplished by advertising a
   replacement route.  The replacement route carries new (changed)
   attributes and has the same address prefix as the original route.

That is the reason.

When in my tests AS65200 looses direct connectivity with AS65100 it sees
AS65300 as a viable path.
It sends a WITHDRAW of the AS65100 prefix to AS65300 via the primary
peering.
On the standby peering no WITHDRAW is sent, instead AS65200 sends an UPDATE
with it's new path. Since this update has AS65300 in the AS-PATH AS65300
will discard
the update and just missed the fact that AS65200 doesn't have connectivity
to AS65100.

Handling an incoming UPDATE with a loop as a WITHDRAW, be it as-path,
cluster-list or
originator-id, sounds pretty good to me right now. I'll sleep  on it and see
how it feels
tomorrow.

As I said, I don't see anything here that violates RFC's, but I have never
seen this before
either. I will try to get the time to check out how IOS and IOS XR handle
this. No point
in re-inventing the wheel if they happen to have a round one.

/Tony

Re: multipath routing with OpenBGPD

[Tony Sarendal]

On 11/3/07, Florian Fuessl [EMAIL PROTECTED] wrote:

 Hi Gregory,

 we have multiple redundant FE upstream peerings to the same AS. So I guess
 the best solution would be in our case to let the upstream provider assign
 different community flags for packets passing each FE line which we can
 use
 for outgoing route preference decisions.

 Other ideas are welcome ;-)


I hope you mean communites on prefixes, and even in that case what can the
provider do that you can't ?

What does your topology look like ?

/Tony

Re: using bgpd and ospfd

[Tony Sarendal]

On 10/30/07, Henning Brauer [EMAIL PROTECTED] wrote:

 * Tony Sarendal [EMAIL PROTECTED] [2007-10-30 02:28]:
  bgp rib and fib look out of sync.
  Any ideas why it behaves this way ?
 
  It seems like the networks that only exist in bgp fail to re-route when
 I
  take
  down a core router that is the current bgp-nexthop.

 lookslike there is a case we miss listening to the routing socket, or
 there is sth in the message that makes us skip it.
 can you run route monitor on the misbehaving machine while causing
 the change and send me the output (no need to spam the list with that
 tho)?



Will do.

So running a setup where ospfd and bgpd carries the same prefixes should
work ?
In the lab setup both ospf and bgp carry the loopback and links, and all
non-core
prefixes are in bgp only.

When I run bgp-only things work like a charm, except for a bit of funkiness
with
existing tcp-sessions to routers showing a bit of funky routing...

/Tony

Re: using bgpd and ospfd

[Tony Sarendal]

On 10/30/07, Henning Brauer [EMAIL PROTECTED] wrote:

 * Tony Sarendal [EMAIL PROTECTED] [2007-10-30 11:25]:
  On 10/30/07, Henning Brauer [EMAIL PROTECTED] wrote:
  
   * Tony Sarendal [EMAIL PROTECTED] [2007-10-30 02:28]:
bgp rib and fib look out of sync.
Any ideas why it behaves this way ?
   
It seems like the networks that only exist in bgp fail to re-route
 when
   I
take
down a core router that is the current bgp-nexthop.
  
   lookslike there is a case we miss listening to the routing socket, or
   there is sth in the message that makes us skip it.
   can you run route monitor on the misbehaving machine while causing
   the change and send me the output (no need to spam the list with that
   tho)?
 
 
 
  Will do.
 
  So running a setup where ospfd and bgpd carries the same prefixes should
  work ?

 oh. the same ones. that is a bit iffy right now.



That was the answer I was looking for =)

No worries, I will adapt the live design I'm implementing as I want it
working
well on 4.2. I can either make sure ospfd and bgpd and don't carry the same
prefixes, which is possibly in this particulare case, or I can go bgp-only.
I'm most likely going bgp-only, hot-potato routing is something I want.

If there is any testing I can do to assist please let me know,
otherwise I'll just continue to play with it. It is my form of mediation
to clear my mind from work.

/Tony

using bgpd and ospfd

[Tony Sarendal]

I set up a test network with bgpd/ospfd, a standard service provider design
where ospf carries the network links and loopbacks and bgp carries
everything,
bgp routers doing nexthop self, core full mesh and access routers rr-clients
of the two nearest core routers.

I'm seeing some pretty odd behaviour that I haven't seen before when only
using bgpd.

Are there any know issues with using this kind of design with bgpd/ospfd ?


Quick example:

View from an access router at another prefix on the other side of the
network
ar1# route get 10.1.102.0
   route to: 10.1.102.0
destination: 10.1.102.0
   mask: 255.255.255.0
gateway: 172.16.1.6
  interface: vlan602
 if address: 172.16.1.5
  flags: UP,GATEWAY,DONE,PROTO1
 use  hopcount   mtuexpire
1470 0 0 0


ar1# bgpctl show fib 10.1.102.0
...
flags destination   gateway
*B10.1.102.0/24 172.16.1.6

ar1# bgpctl show rib 10.1.1.02.0
...
flags destination   gateway lpref   medaspath origin
I*   10.1.102.0/24 192.168.0.1 120 3010   i
I*10.1.102.0/24 192.168.0.2 120 3010   i

ar1# ospfctl show fib 192.168.0.1
flags: * = valid, O = OSPF, C = Connected, S = Static
FLags  Destination  Nexthop
*O 192.168.0.1/32   172.16.1.6

ar1#



So far so good. I now shut down the core router 192.168.0.1
The moment I do that the connectivity dies, even though there is another
path.

ar1# route get 10.1.102.0
   route to: 10.1.102.0
destination: 10.1.102.0
   mask: 255.255.255.0
gateway: 172.16.1.6
  interface: vlan602
 if address: 172.16.1.5
  flags: UP,GATEWAY,DONE,PROTO1
 use  hopcount   mtuexpire
1646 0 0 0

ar1# bgpctl show fib 10.1.102.0
...
flags destination   gateway
*B10.1.102.0/24 172.16.1.6

ar1# bgpctl show rib 10.1.1.02.0
...
flags destination   gateway lpref   medaspath origin
I*10.1.102.0/24 192.168.0.2 120 3010   i

ar1# ospfctl show fib 192.168.0.2
flags: * = valid, O = OSPF, C = Connected, S = Static
FLags  Destination  Nexthop
*O 192.168.0.2/32   172.16.1.2

ar1#


bgp rib and fib look out of sync.
Any ideas why it behaves this way ?

It seems like the networks that only exist in bgp fail to re-route when I
take
down a core router that is the current bgp-nexthop.

/Tony

Re: Moved OpenBSD router from Network A to Network B and Internet no longer works

[Tony Sarendal]

On 10/27/07, Jake Conk [EMAIL PROTECTED] wrote:

 Hello,

 I have my OpenBSD machine setup as a router and when I moved my
 network from my office to my new datacenter I was no longer able to
 connect to the internet from machines behind the obsd router. When I
 try to ping a domain such as google.com from any of the machines
 behind the router I get the ip adress of the domain or host back BUT I
 do not get any successful replies back.

 I do have ipforwarding setup and my openbsd router machine has named
 setup also but as a forwarder to nameservers I have located elsewhere.

 The only thing that changed when moving from network a (the office) to
 network b (the datacenter) was the ip. It use to have a private ip and
 now has a public ip attached to one of the ports. All the internal ips
 with and behind the router remain the same.

 The router has actually 2 public ips, one that is carped and another
 ip address that is just configured as a public ip.

 I don't know what else the problem could be. I've updated my default
 gateway and ip addresses on my openbsd router, what else am I missing
 here? Is there something probably cached that is sending requests from
 my machines behind the router to its old ip that used to be configured
 on the server?

 Please help!


Do your upstream routers know how to find the networks behind your
openbsd router ?

/Tony

Re: Moved OpenBSD router from Network A to Network B and Internet no longer works

[Tony Sarendal]

On 10/27/07, Tony Sarendal [EMAIL PROTECTED] wrote:

 On 10/27/07, Jake Conk [EMAIL PROTECTED] wrote:

  Hello,
 
  I have my OpenBSD machine setup as a router and when I moved my
  network from my office to my new datacenter I was no longer able to
  connect to the internet from machines behind the obsd router. When I
  try to ping a domain such as google.com from any of the machines
  behind the router I get the ip adress of the domain or host back BUT I
  do not get any successful replies back.
 
  I do have ipforwarding setup and my openbsd router machine has named
  setup also but as a forwarder to nameservers I have located elsewhere.
 
  The only thing that changed when moving from network a (the office) to
  network b (the datacenter) was the ip. It use to have a private ip and
  now has a public ip attached to one of the ports. All the internal ips
  with and behind the router remain the same.
 
  The router has actually 2 public ips, one that is carped and another
  ip address that is just configured as a public ip.
 
  I don't know what else the problem could be. I've updated my default
  gateway and ip addresses on my openbsd router, what else am I missing
  here? Is there something probably cached that is sending requests from
  my machines behind the router to its old ip that used to be configured
  on the server?
 
  Please help!


 Do your upstream routers know how to find the networks behind your
 openbsd router ?


I should not send emails before drinking coffee...
You use private addresses on the inside.

Use tcpdump to see that packets going out the firewall are nat'ed correctly,
and the responses come back.

/Tony

openbsd routing and link down

[Tony Sarendal]

I'm testing openbsd and routing in a basic setup.
router-01 and router-02 are access routers with dynamic routing,
both connect to a lan where firewall-01 resides.

Both router-01 and router-02 have a static route for the network
behind firewall-01.

router-01# cat
/etc/hostname.em1
inet 192.168.1.226 255.255.255.224 NONE
! /sbin/ifconfig \$if description FIREWALL LAN
! /sbin/route add 192.168.2.0/24 192.168.1.254

router-02# cat
/etc/hostname.em1
inet 192.168.1.227 255.255.255.224 NONE
! /sbin/ifconfig \$if description FIREWALL LAN
! /sbin/route add 192.168.2.0/24 192.168.1.254

Both routers redistribute statics and connected into bgp.
bgpd.conf:
network connected set localpref 200
network static set localpref 200


In this basic setup, how can I avoid router-01 from black-holing
192.168.1.224/27 and 192.168.2.0/24 if em1 goes down ?

Even in the case of ifconfig em1 down on router-01 I see that
it continues to advertise the connnected and static networks.
The static I can understand as it could resolve via a recursive lookup,
if openbsd does that.

Regards Tony

Re: em(4) - IFCAP_VLAN_MTU IFCAP_VLAN_HWTAGGING ?

[Tony Sarendal]

On 10/23/07, Henning Brauer [EMAIL PROTECTED] wrote:

 * Tony Sarendal [EMAIL PROTECTED] [2007-10-22 18:33]:
  I didn't get that opinion from marketing.
  No matter, we disagree, lets leave it at that.

 well, yeah, nontheless, I wanna point out the essence why stateful is
 better (the way we do it in OpenBSD):

 1) it moves the limit where the box starts to suffer from overload quite
far, or, in other words, the box can handle a much larger amount of
traffic before it starts to drop stuff. thus it can withstand bigger
amounts of (D)DoS too.
 2) once it gets to that point, it is more selective in dropping packets
than a stateless box, as it prefers established connections. this
behaviour cannot be valued enough in (D)DoS type of situations.


I wish to implement things in a way where the link is the limitation,
not the box. But there is no point in re-doing that discussion.

When I have some time free I'll test it in the lab to see that difference in
behaviour. Any ideas of when you will get around to handling assymetric
traffic in a stateful way ?

/Tony

Re: em(4) - IFCAP_VLAN_MTU IFCAP_VLAN_HWTAGGING ?

[Tony Sarendal]

On 10/23/07, ropers [EMAIL PROTECTED] wrote:

 On 23/10/2007, Tony Sarendal [EMAIL PROTECTED] wrote:
  On 10/23/07, Henning Brauer [EMAIL PROTECTED] wrote:
  
   * Tony Sarendal [EMAIL PROTECTED] [2007-10-22 18:33]:
I didn't get that opinion from marketing.
No matter, we disagree, lets leave it at that.
  
   well, yeah, nontheless, I wanna point out the essence why stateful is
   better (the way we do it in OpenBSD):
  
   1) it moves the limit where the box starts to suffer from overload
 quite
  far, or, in other words, the box can handle a much larger amount of
  traffic before it starts to drop stuff. thus it can withstand
 bigger
  amounts of (D)DoS too.
   2) once it gets to that point, it is more selective in dropping
 packets
  than a stateless box, as it prefers established connections. this
  behaviour cannot be valued enough in (D)DoS type of situations.
 
 
  I wish to implement things in a way where the link is the limitation,
  not the box. But there is no point in re-doing that discussion.
 
  When I have some time free I'll test it in the lab to see that
 difference in
  behaviour.

 I know very little, but I would like to note that some providers (
 http://www.rayservers.com/ddos-protection ) deploy OpenBSD with the
 express purpose of offering dDoS protection. That has to count for
 something.

 OTOH, Henning's word alone would be enough for me, because AFAIK
 Henning wrote actual pertinent code and knows darn friggin well what
 he's talking about. Did you contribute as much code to OpenBSD/pf as
 Henning? Are you sure your understanding is deeper than his? (No
 offense, by the way, all in good humour.)


Henning has committed more code than me. If you count in percent
infinetly more. Does that mean that I don't know what I'm talking about ?

I use OpenBSD because I like it, I think it is the best project I can find
on the net.
I don't belive a fan-boy attitude is an asset to the project, that is what
you
are contributing right now.

This is a view of the a external peering link where I work now:
  5 minute input rate 6165205000 bits/sec, 1036946 packets/sec
  5 minute output rate 3134466000 bits/sec, 1000242 packets/sec
One link out of many, no DDOS going on. Maybe I should stick a rayserver on
it.

Correct me if I'm wrong, but Henning needs someone to argue with him and
pester him.

/Tony

Re: em(4) - IFCAP_VLAN_MTU IFCAP_VLAN_HWTAGGING ?

[Tony Sarendal]

On 10/22/07, Henning Brauer [EMAIL PROTECTED] wrote:

 * Tony Sarendal [EMAIL PROTECTED] [2007-10-22 01:19]:
  On 10/21/07, Henning Brauer [EMAIL PROTECTED] wrote:
   well, you can go stateful up to a certain point and handle stuff above
   stateless (better than dropping), like
  
   pass out on X from $foo
   pass in  on X to $foo
   pass out on X from $foo keep state(max 1)
 
 
  To design a reliable IP network I would need the devices to be able to
  handle
  the desired pps rate even when that state limit is exceeded.

 so? where is the contradiction here?


No contradiction. If the requirement is to be wirespeed the forwarding
performance under ideal conditions is not relevant.


 Many routing devices have over the years achieved good performance by
  different flow caching
  methods, we have over the years also learnt that this is a bad thing in
  uncontrolled environments
  like the Internet.

 no, that is entirely bullshit, sorry.

 if flow cahcing allows your device to work more efficient in the usual
 case, hey, excellent, you would be dumb to not use it.

 this does NOT save you from either leaving enough headroom that you can
 heandle the packet rate when exceeding your state limit or at least
 know about and live with the limitation.



A Cisco6509 SUPA1/MSFC2 could do around 10Mpps under normal conditions,
but not even 500kpps when flow count exceeded what it could handle
in hardware. Good boxes for the internal network, horrible for the
datacenter or internet core/edge.

Are the 10Mpps they can do relevant if the policy states all devices
should be be wirespeed ? If we were to use them would we enable the
mls switching anyway? Probably.


 A reliable IP router is wirespeed and stateless. There is no getting
 around
  that.

 oh really.
 I say it is bullshit


Are you officially stating that the added complexity of stateful forwarding
does not increase the likelyhood of unpredictable behaviour ?

.
 there is no single wirespeed in all circumstances router on the market,
 not even for fast ethernet. that is a marketing gag. a 10 MBit/s stream
 of correctly and purposefully craftet packets brings each and every
 router you can buy to its knees. if it works like an OpenBSD machine
 with stateful filters which prefers established states in the overload
 case, it doesn't suffer as badly as the stateless ones.



Something as simple as being able to forward packets independently
of the source/destination pattern and protocol hardly qualifies as
the specific/unknown case where you can make a 80Mpps per line card
CRS-1's not even forward 10Mbps.

OpenBSD once shipped with a remote root compromise, this was addressed.
When we find new scenarios that can prevent the routers from performing
as expected we try to address that. There will always be unknown corner
cases showing up, and that we need to handle. We do not give up
and go out and buy Ford Pinto's just because there is a possibility of
a new Mercedes blowing up from a slight nudge from behind.

No need to get aggressive, Henning.
I don't agree with you. I say that a stateless device in general is more
reliable than a stateful one.

Regards Tony

Re: em(4) - IFCAP_VLAN_MTU IFCAP_VLAN_HWTAGGING ?

[Tony Sarendal]

On 10/22/07, Henning Brauer [EMAIL PROTECTED] wrote:

 * Tony Sarendal [EMAIL PROTECTED] [2007-10-22 14:59]:
  On 10/22/07, Henning Brauer [EMAIL PROTECTED] wrote:
   * Tony Sarendal [EMAIL PROTECTED] [2007-10-22 01:19]:
On 10/21/07, Henning Brauer [EMAIL PROTECTED] wrote:
 well, you can go stateful up to a certain point and handle stuff
 above
 stateless (better than dropping), like

 pass out on X from $foo
 pass in  on X to $foo
 pass out on X from $foo keep state(max 1)
To design a reliable IP network I would need the devices to be able
 to
handle
the desired pps rate even when that state limit is exceeded.
   so? where is the contradiction here?
  No contradiction. If the requirement is to be wirespeed the forwarding
  performance under ideal conditions is not relevant.

 with the amount of states you can handle, I don't think it is a limit
 very relevant in practice. Or, in other words, if you need to handle so
 many more flows than we can handle statefully, you are at a point where
 you cannot realisticly use a commodity hardware router any more.

   Many routing devices have over the years achieved good performance by
different flow caching
methods, we have over the years also learnt that this is a bad thing
 in
uncontrolled environments
like the Internet.
   no, that is entirely bullshit, sorry.
  
   if flow cahcing allows your device to work more efficient in the usual
   case, hey, excellent, you would be dumb to not use it.
  
   this does NOT save you from either leaving enough headroom that you
 can
   heandle the packet rate when exceeding your state limit or at least
   know about and live with the limitation.
  A Cisco6509 SUPA1/MSFC2 could do around 10Mpps under normal conditions,
  but not even 500kpps when flow count exceeded what it could handle
  in hardware. Good boxes for the internal network, horrible for the
  datacenter or internet core/edge.

 and I bet I can make up a 10 or maybe 100 Kpps stream that makes it fall
 over.

   A reliable IP router is wirespeed and stateless. There is no getting
   around
that.
  
   oh really.
   I say it is bullshit
  Are you officially stating that the added complexity of stateful
 forwarding
  does not increase the likelyhood of unpredictable behaviour ?

 yes. the state tracking is not THAT difficult and very very very mature.

   there is no single wirespeed in all circumstances router on the
 market,
   not even for fast ethernet. that is a marketing gag. a 10 MBit/s
 stream
   of correctly and purposefully craftet packets brings each and every
   router you can buy to its knees. if it works like an OpenBSD machine
   with stateful filters which prefers established states in the overload
   case, it doesn't suffer as badly as the stateless ones.
  Something as simple as being able to forward packets independently
  of the source/destination pattern and protocol hardly qualifies as
  the specific/unknown case where you can make a 80Mpps per line card
  CRS-1's not even forward 10Mbps.

 i can't parse what you wanna say here.

  OpenBSD once shipped with a remote root compromise, this was addressed.
  When we find new scenarios that can prevent the routers from performing
  as expected we try to address that. There will always be unknown corner
  cases showing up, and that we need to handle.

 which is totally independent from specific implementations. this is
 true for each and every piece of hard  software available.

  No need to get aggressive, Henning.

 I'm not aggressive :)

  I don't agree with you. I say that a stateless device in general is more
  reliable than a stateful one.

 and I say that is totally poop. It is a marketing lie.


I didn't get that opinion from marketing.
No matter, we disagree, lets leave it at that.

/Tony

Re: em(4) - IFCAP_VLAN_MTU IFCAP_VLAN_HWTAGGING ?

[Tony Sarendal]

On 10/21/07, Henning Brauer [EMAIL PROTECTED] wrote:

 * Tony Sarendal [EMAIL PROTECTED] [2007-10-20 18:06]:
  On 10/20/07, Henning Brauer [EMAIL PROTECTED] wrote:
  
   * Tony Sarendal [EMAIL PROTECTED] [2007-10-20 13:24]:
Once I have a few moments free I'll check the impact of pf with urpf
 and
basic stateless filters
filters enabled. Time to go look for a light sabre for my son.
  
   stateless filters? why oh why? they're SLOWER than stateful. far.
 
 
  Stateful filters on an internet router  does not seem like a very good
  idea to me. Traffic may exit and enter on different devices, it is
 another
  limited resource, and it adds another layer of complexity.

 well, we need a knob for lose state tracking to alow these assymetric
 routing scenarios, it is on my agenda.
 otherwise, either no filter at all or stteful. stateless is poop.



What will happen when the limit of maximum concurrent states is reached ?
Will it stop forwarding new flows ?

/Tony

Re: em(4) - IFCAP_VLAN_MTU IFCAP_VLAN_HWTAGGING ?

[Tony Sarendal]

On 10/21/07, Henning Brauer [EMAIL PROTECTED] wrote:

 * Tony Sarendal [EMAIL PROTECTED] [2007-10-21 14:50]:
   stateless is poop.
  What will happen when the limit of maximum concurrent states is reached
 ?
  Will it stop forwarding new flows ?

 depends on the way you write your ruleset.
 if you do nothing, exactly that happens.


An incoming packet is either dropped or not, I don't see how the router can
do nothing.

Besides that, the environment I am looking at is as an edge/peering router.
Basic filtering to protect infrastructure and where possible prevent
spoofing,
I do not consider such an environment a suitable place for a statelful
device
as they normally change behaviour when the limit of states is exceeded.

A router that has a major performance drop when a certain limit of flows is
reached is something I normally stay away from, a router that stops
forwarding of new flows when a flow limit is reached is worse.

That is my reasoning for using stateless filters in my case.
If OpenBSD/pf has a solution that solves these stateful limitations I would
be
very interested in understanding it.

/Tony

Re: em(4) - IFCAP_VLAN_MTU IFCAP_VLAN_HWTAGGING ?

[Tony Sarendal]

On 10/21/07, Can Erkin Acar [EMAIL PROTECTED] wrote:

 Tony Sarendal [EMAIL PROTECTED] wrote:
  On 10/21/07, Henning Brauer [EMAIL PROTECTED] wrote:
 
  * Tony Sarendal [EMAIL PROTECTED] [2007-10-21 14:50]:
stateless is poop.
   What will happen when the limit of maximum concurrent states is
 reached
  ?
   Will it stop forwarding new flows ?
 
  depends on the way you write your ruleset.
  if you do nothing, exactly that happens.
 
 
  An incoming packet is either dropped or not, I don't see how the router
 can
  do nothing.
 
  Besides that, the environment I am looking at is as an edge/peering
 router.
  Basic filtering to protect infrastructure and where possible prevent
  spoofing,
  I do not consider such an environment a suitable place for a statelful
  device
  as they normally change behaviour when the limit of states is exceeded.
 
  A router that has a major performance drop when a certain limit of flows
 is
  reached is something I normally stay away from, a router that stops
  forwarding of new flows when a flow limit is reached is worse.
 
  That is my reasoning for using stateless filters in my case.
  If OpenBSD/pf has a solution that solves these stateful limitations I
 would
  be
  very interested in understanding it.

 There are several mechanisms for handling overload in OpenBSD/pf
 They are designed to improve the stateful filtering experience, and
 usually
 non-stateful traffic suffers in case of overload.

 One of them is adaptive timeouts. This lets 'old/not-established' states
 to be expired quickly, letting new states to be established. You need to
 set suitable values for your traffic requirements, together with max
 number of
 states. This is much more useful than a best effort forwarding where
 packets
 belonging to valid states are equally likely to be dropped.

 Another mechanism is congestion handling. If an interface input queue is
 full,
 that means your CPU is not able to keep up with the incoming packets (or
 that
 you need to increase the queue size). In this case, pf stops ruleset
 evaluation
 altogether, and ONLY passes packets matching to states. Note that, in
 this case
 we are already dropping packets. Having pf choose only stateful traffic
 to pass
 is an improvement over not being able to pass any.

 Considering the fact that state matching is faster per packet when you
 have
 some less-than-trivial ruleset. Using stateful filtering is better in
 almost
 all cases (ie. except asymmetric routing as you pointed out earlier).



The last sentence sums up what I'm looking at.
Although I don't feel that assymetric routing is an exception.

/Tony

Re: em(4) - IFCAP_VLAN_MTU IFCAP_VLAN_HWTAGGING ?

[Tony Sarendal]

On 10/21/07, Henning Brauer [EMAIL PROTECTED] wrote:

 * Tony Sarendal [EMAIL PROTECTED] [2007-10-21 17:22]:
  On 10/21/07, Henning Brauer [EMAIL PROTECTED] wrote:
  
   * Tony Sarendal [EMAIL PROTECTED] [2007-10-21 14:50]:
 stateless is poop.
What will happen when the limit of maximum concurrent states is
 reached
   ?
Will it stop forwarding new flows ?
  
   depends on the way you write your ruleset.
   if you do nothing, exactly that happens.
 
 
  An incoming packet is either dropped or not, I don't see how the router
 can
  do nothing.

 you misunderstood; if you do nothing to prevent that situation what you
 described happens.

  Besides that, the environment I am looking at is as an edge/peering
 router.
  Basic filtering to protect infrastructure and where possible prevent
  spoofing,
  I do not consider such an environment a suitable place for a statelful
  device
  as they normally change behaviour when the limit of states is exceeded.
 
  A router that has a major performance drop when a certain limit of flows
 is
  reached is something I normally stay away from, a router that stops
  forwarding of new flows when a flow limit is reached is worse.
 
  That is my reasoning for using stateless filters in my case.
  If OpenBSD/pf has a solution that solves these stateful limitations I
 would
  be
  very interested in understanding it.

 well, you can go stateful up to a certain point and handle stuff above
 stateless (better than dropping), like

 pass out on X from $foo
 pass in  on X to $foo
 pass out on X from $foo keep state(max 1)


To design a reliable IP network I would need the devices to be able to
handle
the desired pps rate even when that state limit is exceeded.

Many routing devices have over the years achieved good performance by
different flow caching
methods, we have over the years also learnt that this is a bad thing in
uncontrolled environments
like the Internet.

A reliable IP router is wirespeed and stateless. There is no getting around
that.
In my case I would verify that the box is wirespeed in the environment I put
it in, the fact that
it can be faster under certain conditions is less interesting.

/Tony

Re: em(4) - IFCAP_VLAN_MTU IFCAP_VLAN_HWTAGGING ?

[Tony Sarendal]

I performed some quick additional tests with OpenBSD and vlan's just
for the fun of it, although I belive these tests were more about OpenBSD's
performance with lots of interfaces.

If you want a openbsd router/firewall with 4000 interfaces don't go for a
low-end CPU =)

http://www.layer17.net/openbsd-test-vlan-quick.html

/Tony

Re: em(4) - IFCAP_VLAN_MTU IFCAP_VLAN_HWTAGGING ?

[Tony Sarendal]

On 10/20/07, Henning Brauer [EMAIL PROTECTED] wrote:

 * Tony Sarendal [EMAIL PROTECTED] [2007-10-20 09:49]:
  I performed some quick additional tests with OpenBSD and vlan's just
  for the fun of it, although I belive these tests were more about
 OpenBSD's
  performance with lots of interfaces.
 
  If you want a openbsd router/firewall with 4000 interfaces don't go for
 a
  low-end CPU =)
 
  http://www.layer17.net/openbsd-test-vlan-quick.html

 right, we're not too efficient with lots of lots of interfaces. on the
 agenda...


The box can in it's current state be a solid 100M router with 50
interfaces+uplinks on it.

Once I have a few moments free I'll check the impact of pf with urpf and
basic stateless filters
filters enabled. Time to go look for a light sabre for my son.

/Tony

Re: em(4) - IFCAP_VLAN_MTU IFCAP_VLAN_HWTAGGING ?

[Tony Sarendal]

On 10/20/07, Henning Brauer [EMAIL PROTECTED] wrote:

 * Tony Sarendal [EMAIL PROTECTED] [2007-10-20 13:24]:
  Once I have a few moments free I'll check the impact of pf with urpf and
  basic stateless filters
  filters enabled. Time to go look for a light sabre for my son.

 stateless filters? why oh why? they're SLOWER than stateful. far.


Stateful filters on an internet router  does not seem like a very good
idea to me. Traffic may exit and enter on different devices, it is another
limited resource, and it adds another layer of complexity.

/Tony

Re: Idle sessions dying on crappy router: How to increase TCP keepalive?

[Tony Sarendal]

On 10/20/07, Timo Schoeler [EMAIL PROTECTED] wrote:

 Hi list,

 on a customers' site I have a problem connecting from within their
 LAN (OpenBSD machine) crossing their router (Linksys BEFSX41, doing
 NAT) to a machine on the internet via SSH: Sessions die after some time
 due to 'timeouts'.

 If the connection is not used heavily (e.g. showing top(1)) it dies
 (the router clearing it's session cache); it's a well-known issue with
 this kind of customer-class devices (lots of entries on your favorite
 search engine).

 A solution (for GNU/Linux) would be to increase

 /proc/sys/net/ipv4/tcp_keepalive_time

 as I got from a newsgroup; however, on OpenBSD I just see

 net.inet.tcp.keepinittime
 net.inet.tcp.keepidle
 net.inet.tcp.keepintvl

 I tried to increase (and decrease, just to determine if there's any
 difference) net.inet.tcp.keepidle, but it didn't make a difference.
 Think I'm using the wrong knob -- is there something similar on OpenBSD
 (like tcp_keepalive_time) to cheat on the NAT thing?

 (And, yes, using a WRAP board running OpenBSD as router works
 perfectly well in the same environment; however, the Linksys has to
 stay there...)

 TIA,

 Timo


You can ask ssh to do keepalives for you.
Look at the ServerAliveInterval and ClientAliveInterval in ssh.

/Tony

Re: em(4) - IFCAP_VLAN_MTU IFCAP_VLAN_HWTAGGING ?

[Tony Sarendal]

On 10/18/07, Brian A. Seklecki [EMAIL PROTECTED] wrote:

 On Wed, 17 Oct 2007 10:52:34 +0200
 Henning Brauer [EMAIL PROTECTED] wrote:

  * Brian A. Seklecki [EMAIL PROTECTED] [2007-10-16
 23:01]:
   All:
  
   I see that IFCAP_VLAN_MTU is available, but IFCAP_VLAN_HWTAGGING, as
 seen
   in ti(4), is absent in em(4).  Version 6.6.6 of em(4) elsewhere is
   promising TOE (TCP Segment Offload) and already supports a vlanhwtag
 and
   jumbo frames.
  
   For VLAN routing security boxes, this would be a big plus for a lot of
   embedded SBCs that support only integrated Intel NICs.
 
  this would be a big plus?
  sez who?

 Says my assumption that unlike TOE, VLAN hardware tagging has been well
 accepted and established for some time.  But only an objective test can
 tell...

 I've been looking for details on an F/OSS L2-L4 Performance Benchmark
 System.

 Someone on the lists recently posted test results of tests using IXIA

 http://www.ixiacom.com/

 I don't know where I'm going to find the time, but I will eventually have
 a go at this.


One box with two em interfaces.
One 175kpps stream in each direction, 64 byte frame size.

Both interfaces with normal config = ~20% idle cpu
One interface converted to VLAN trunk with one vlan = ~15% idle cpu

No packet loss.

Just a 5 minute quick test, nothing too scientific.

/Tony

Re: em(4) - IFCAP_VLAN_MTU IFCAP_VLAN_HWTAGGING ?

[Tony Sarendal]

On 10/18/07, Brian A. Seklecki [EMAIL PROTECTED] wrote:

 On Thu, 18 Oct 2007 14:16:59 +0100
 Tony Sarendal [EMAIL PROTECTED] wrote:

  Just a 5 minute quick test, nothing too scientific.

 Thanks!  What was your IXIA platform?  RHEL with gig interface or an
 appliance?


I used an Optixia XM12, at least thats what I think it's called.

/Tony

OpenBSD router performance tests

[Tony Sarendal]

I made a new more detailed latency/throughput test with ifq.maxlen set to
2500. With AMD64 UP kernel we are now looking at around 500kpps
without packet loss. From 400 to 500kpps with one command, pretty nice,
I have to remember that one.
http://www.layer17.net/openbsd-test-rfc2544-throughput-latency.html (bottom
of the page).

Next up is the i386 kernel.

/Tony

Re: Speed Problems

[Tony Sarendal]

On 10/3/07, Claudio Jeker [EMAIL PROTECTED] wrote:

 On Tue, Oct 02, 2007 at 08:46:43PM +0100, Tony Sarendal wrote:
  On 9/27/07, Tony Sarendal [EMAIL PROTECTED] wrote:
  
   On 9/27/07, Claudio Jeker [EMAIL PROTECTED] wrote:

 ...

 
  I hooked up the X4100 to one of our testers and ran some basic tests
 just to
  get
  familiar with the tester.
 
  I put up the results of the first run of tests on
  http://www.layer17.net/openbsd-router-intro.html
 
  All opinions are welcome, please be gentle.
 
  I hope to be able to test the 1k vlan interface firewall setup later,
  I just need to baseline a bit first.
 

 Quite interesting numbers. I guess that em(4) does still to many pci
 read/write accesses and so 64byte packet storms are mostly limited by the
 PCI bus access delay. My gut feeling is that the TX path is causing the
 slow down (enqueing happens on a per packet basis and that is porbably not
 optimal for current high speed cards).
 Could you add the dmesg of the test box to the website?
 Do you have any other network cards you could test? (I'm mostly interested
 in bnx but sk, msk, bge and nfe could be interesting as well).


I'll put up the dmesg when I'm in the office again.
The nfe port I do management over has jammed, a little bios tweaking
might fix that.

The only cards I have access to at the moment are the builtins,
2xem and 2xnfe.

The packet drops of 64 byte frames in the throughput/latency test is a bit
confusing, I can't see that behaviour if I slowly ramp up from 1kpps.
Before I do tests with more advanced config I want the basic ones to give
a result I understand so I'll try to figure that one out.

/Tony

Re: Speed Problems

[Tony Sarendal]

On 10/3/07, Daniel Ouellet [EMAIL PROTECTED] wrote:

 Claudio Jeker wrote:
  Could you add the dmesg of the test box to the website?
  Do you have any other network cards you could test? (I'm mostly
 interested
  in bnx but sk, msk, bge and nfe could be interesting as well).

 This box if the M2 version also come with nfe cards as well, but there
 is issue with it at the moment. dmesg available:

 http://cvs.openbsd.org/cgi-bin/query-pr-wrapper?full=yesnumbers=5587


Dmesg's are on the site now.
http://www.layer17.net/openbsd-test-setup.html

Note that the box actually has 8Gigs of memory.

Since I'm off-site I had to get someone else to powercycle the box for me
to wake up the nfe I use as management interface, so the MP dmesg is
from the logs.

Running with the SP kernel the nfe's seem to work ok.

I'm running the same set of tests with the SP kernel right now.
The 64 byte frames issue in the throughput/latency test looks to be gone...
cross fingers...

/Tony




Daniel

Re: Speed Problems

[Tony Sarendal]

New set of tests done with AMD64 UP kernel.

http://www.layer17.net/openbsd-router-intro.html

/Tony

Re: Speed Problems

[Tony Sarendal]

On 9/27/07, Tony Sarendal [EMAIL PROTECTED] wrote:

 On 9/27/07, Claudio Jeker [EMAIL PROTECTED] wrote:

  On Thu, Sep 27, 2007 at 09:54:00AM +0100, Tony Sarendal wrote:
   On 9/27/07, Henning Brauer [EMAIL PROTECTED] wrote:
   
* Tony Sarendal  [EMAIL PROTECTED] [2007-09-27 10:36]:
 On 9/26/07, Tom Bombadil [EMAIL PROTECTED] wrote:
   net.inet.ip.ifq.maxlen defines how many packets can be queued
  in the
IP
   input queue before further packets are dropped. Packets
  comming from
the
   network card are first put into this queue and the actuall IP
  packet
   processing is done later. Gigabit cards with interrupt
  mitigation
may
  spit
   out many packets per interrupt plus heavy use of pf can
  slowdown the
   packet forwarding. So it is possible that a heavy burst of
  packets
is
   overflowing this queue. On the other hand you do not want to
  use a
too
  big
   number because this has negative effects on the system
  (livelock
etc).
   256 seems to be a better default then the 50 but additional
  tweaking
may
   allow you to process a few packets more.
  Thanks Claudio...
  In the link that Stuart posted here, Henning mentions 256 times
  the
  number of interfaces:
  http://archive.openbsd.nu/?ml=openbsd-techa=2006-10t=2474666
 Is that per physical or per logical interface  ?
   
it is a rule of thumb. an approximation. for typical cases.
   
 [EMAIL PROTECTED] ifconfig -a | grep ^vlan | wc -l
 4094
   
that is not a typical case.
you do not wanna set your ifqlen to 1048064 :)
   
the highest qlen I have is somewhere around 2500.
where the high watermark is... I cannot really say. I'd be careful
going far higher than the above.
  
  
  
   I meant if the input queue length was per physical or logical
  interface.
   There are places where I actually need boxes with more than 1k vlan
   subinterfaces.
   If net.inet.ip.ifq.maxlen is per logical interface I see some
  potentional
   issues under load.
  
 
  Henning's hint of 256 * num of interfaces is for physical interfaces.
  The virtual interfaces will just see a subset of the packets comming
  from
  the real ones and so they can be ignored in that rule of thumb.
 
  Do you have systems with 1000 and more interfaces in production?
  Any performance issues? Many interface related operations are O(N).
  Fixing this is another item on my network stack todo list -- as usual
  feel
  free to send me diffs :)


 It's still in design/test phase. I'm going to use an Ixia tester and an
 X4100
 if I find the time to test it, this is a little pet project of my own.
 If I get that far I'll let you know.

 /Tony


I hooked up the X4100 to one of our testers and ran some basic tests just to
get
familiar with the tester.

I put up the results of the first run of tests on
http://www.layer17.net/openbsd-router-intro.html

All opinions are welcome, please be gentle.

I hope to be able to test the 1k vlan interface firewall setup later,
I just need to baseline a bit first.

/Tony

Re: Speed Problems

[Tony Sarendal]

On 9/26/07, Tom Bombadil [EMAIL PROTECTED] wrote:

  net.inet.ip.ifq.maxlen defines how many packets can be queued in the IP
  input queue before further packets are dropped. Packets comming from the
  network card are first put into this queue and the actuall IP packet
  processing is done later. Gigabit cards with interrupt mitigation may
 spit
  out many packets per interrupt plus heavy use of pf can slowdown the
  packet forwarding. So it is possible that a heavy burst of packets is
  overflowing this queue. On the other hand you do not want to use a too
 big
  number because this has negative effects on the system (livelock etc).
  256 seems to be a better default then the 50 but additional tweaking may
  allow you to process a few packets more.

 Thanks Claudio...

 In the link that Stuart posted here, Henning mentions 256 times the
 number of interfaces:
 http://archive.openbsd.nu/?ml=openbsd-techa=2006-10t=2474666


Is that per physical or per logical interface  ?

[EMAIL PROTECTED] ifconfig -a | grep ^vlan | wc -l
4094
[EMAIL PROTECTED]

/Tony

Re: Speed Problems

[Tony Sarendal]

On 9/27/07, Henning Brauer [EMAIL PROTECTED] wrote:

 * Tony Sarendal [EMAIL PROTECTED] [2007-09-27 10:36]:
  On 9/26/07, Tom Bombadil [EMAIL PROTECTED] wrote:
net.inet.ip.ifq.maxlen defines how many packets can be queued in the
 IP
input queue before further packets are dropped. Packets comming from
 the
network card are first put into this queue and the actuall IP packet
processing is done later. Gigabit cards with interrupt mitigation
 may
   spit
out many packets per interrupt plus heavy use of pf can slowdown the
packet forwarding. So it is possible that a heavy burst of packets
 is
overflowing this queue. On the other hand you do not want to use a
 too
   big
number because this has negative effects on the system (livelock
 etc).
256 seems to be a better default then the 50 but additional tweaking
 may
allow you to process a few packets more.
   Thanks Claudio...
   In the link that Stuart posted here, Henning mentions 256 times the
   number of interfaces:
   http://archive.openbsd.nu/?ml=openbsd-techa=2006-10t=2474666
  Is that per physical or per logical interface  ?

 it is a rule of thumb. an approximation. for typical cases.

  [EMAIL PROTECTED] ifconfig -a | grep ^vlan | wc -l
  4094

 that is not a typical case.
 you do not wanna set your ifqlen to 1048064 :)

 the highest qlen I have is somewhere around 2500.
 where the high watermark is... I cannot really say. I'd be careful
 going far higher than the above.



I meant if the input queue length was per physical or logical interface.
There are places where I actually need boxes with more than 1k vlan
subinterfaces.
If net.inet.ip.ifq.maxlen is per logical interface I see some potentional
issues under load.

/Tony

Re: Speed Problems

[Tony Sarendal]

On 9/27/07, Henning Brauer [EMAIL PROTECTED] wrote:

 * Tony Sarendal [EMAIL PROTECTED] [2007-09-27 10:59]:
  I meant if the input queue length was per physical or logical interface.

 neither. there is one per protocol. i. e. typically two (inet and
 inet6).


Very good. My preconfigured firewalls with 4k interfaces, urpf and
stateless rules may actually work in live conditions then.

I'll see if I can hit it with a tester to see what performance I get.

/Tony