On Mon, 4 Mar 2019, 09:43 Tony Sarendal, <t...@polarcap.org> wrote: > > > Den mån 4 mars 2019 kl 09:26 skrev Tony Sarendal <t...@polarcap.org>: > >> Den sön 3 mars 2019 kl 21:35 skrev Theo de Raadt <dera...@openbsd.org>: >> >>> Tony, >>> >>> Are you out of your mind? You didn't provide even a rough hint about >>> what your firewall configuration looks like. You recognize that's >>> pathetic, right? >>> >>> > Earlier in the week I could run parallel ping-pong tests through my >>> test >>> > firewalls >>> > at 300kpps without any packet loss. I updated to the latest snapshot >>> today >>> > and >>> > start to see packet loss at around 80kpps. >>> > >>> > /T >>> > >>> > OpenBSD 6.5-beta (GENERIC.MP) #764: Sun Mar 3 10:24:08 MST 2019 >>> > dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/ >>> GENERIC.MP >>> > real mem = 34300891136 (32711MB) >>> > avail mem = 33251393536 (31711MB) >>> > mpath0 at root >>> > scsibus0 at mpath0: 256 targets >>> > mainbus0 at root >>> > bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xec170 (34 entries) >>> > bios0: vendor American Megatrends Inc. version "3.0" date 04/24/2015 >>> > bios0: Supermicro X10SLD >>> > acpi0 at bios0: rev 2 >>> > acpi0: sleep states S0 S4 S5 >>> > acpi0: tables DSDT FACP APIC FPDT FIDT SSDT SSDT MCFG PRAD HPET SSDT >>> SSDT >>> > SPMI DMAR EINJ ERST HEST BERT >>> > acpi0: wakeup devices PEGP(S4) PEG0(S4) PEGP(S4) PEG1(S4) PEGP(S4) >>> PEG2(S4) >>> > PXSX(S4) RP01(S4) PXSX(S4) RP02(S4) PXSX(S4) RP03(S4) PXSX(S4) RP04(S4) >>> > PXSX(S4) RP05(S4) [...] >>> > acpitimer0 at acpi0: 3579545 Hz, 24 bits >>> > acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat >>> > cpu0 at mainbus0: apid 0 (boot processor) >>> > cpu0: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.68 MHz, 06-3c-03 >>> > cpu0: >>> > >>> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN >>> > cpu0: 256KB 64b/line 8-way L2 cache >>> > cpu0: smt 0, core 0, package 0 >>> > mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges >>> > cpu0: apic clock running at 99MHz >>> > cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4, IBE >>> > cpu1 at mainbus0: apid 2 (application processor) >>> > cpu1: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.01 MHz, 06-3c-03 >>> > cpu1: >>> > >>> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN >>> > cpu1: 256KB 64b/line 8-way L2 cache >>> > cpu1: smt 0, core 1, package 0 >>> > cpu2 at mainbus0: apid 4 (application processor) >>> > cpu2: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.01 MHz, 06-3c-03 >>> > cpu2: >>> > >>> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN >>> > cpu2: 256KB 64b/line 8-way L2 cache >>> > cpu2: smt 0, core 2, package 0 >>> > cpu3 at mainbus0: apid 6 (application processor) >>> > cpu3: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.01 MHz, 06-3c-03 >>> > cpu3: >>> > >>> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN >>> > cpu3: 256KB 64b/line 8-way L2 cache >>> > cpu3: smt 0, core 3, package 0 >>> > ioapic0 at mainbus0: apid 8 pa 0xfec00000, version 20, 24 pins >>> > acpimcfg0 at acpi0 >>> > acpimcfg0: addr 0xf8000000, bus 0-63 >>> > acpihpet0 at acpi0: 14318179 Hz >>> > acpiprt0 at acpi0: bus 0 (PCI0) >>> > acpiprt1 at acpi0: bus 1 (PEG0) >>> > acpiprt2 at acpi0: bus 2 (PEG1) >>> > acpiprt3 at acpi0: bus -1 (PEG2) >>> > acpiprt4 at acpi0: bus 3 (RP01) >>> > acpiprt5 at acpi0: bus -1 (RP02) >>> > acpiprt6 at acpi0: bus -1 (RP03) >>> > acpiprt7 at acpi0: bus -1 (RP04) >>> > acpiprt8 at acpi0: bus -1 (RP05) >>> > acpiprt9 at acpi0: bus -1 (RP06) >>> > acpiprt10 at acpi0: bus -1 (RP07) >>> > acpiprt11 at acpi0: bus -1 (RP08) >>> > acpiec0 at acpi0: not present >>> > acpicpu0 at acpi0: C1(@1 halt!) >>> > acpicpu1 at acpi0: C1(@1 halt!) >>> > acpicpu2 at acpi0: C1(@1 halt!) >>> > acpicpu3 at acpi0: C1(@1 halt!) >>> > acpipwrres0 at acpi0: PG00, resource for PEG0 >>> > acpipwrres1 at acpi0: PG01, resource for PEG1 >>> > acpipwrres2 at acpi0: PG02, resource for PEG2 >>> > acpipwrres3 at acpi0: FN00, resource for FAN0 >>> > acpipwrres4 at acpi0: FN01, resource for FAN1 >>> > acpipwrres5 at acpi0: FN02, resource for FAN2 >>> > acpipwrres6 at acpi0: FN03, resource for FAN3 >>> > acpipwrres7 at acpi0: FN04, resource for FAN4 >>> > acpitz0 at acpi0: critical temperature is 105 degC >>> > acpitz1 at acpi0: critical temperature is 105 degC >>> > acpipci0 at acpi0 PCI0: 0x00000000 0x00000011 0x00000001 >>> > acpicmos0 at acpi0 >>> > "IPI0001" at acpi0 not configured >>> > acpibtn0 at acpi0: SLPB >>> > acpibtn1 at acpi0: PWRB >>> > "PNP0C0B" at acpi0 not configured >>> > "PNP0C0B" at acpi0 not configured >>> > "PNP0C0B" at acpi0 not configured >>> > "PNP0C0B" at acpi0 not configured >>> > "PNP0C0B" at acpi0 not configured >>> > acpivideo0 at acpi0: GFX0 >>> > acpivout0 at acpivideo0: DD1F >>> > ipmi at mainbus0 not configured >>> > pci0 at mainbus0 bus 0 >>> > pchb0 at pci0 dev 0 function 0 "Intel Xeon E3-1200 v3 Host" rev 0x06 >>> > ppb0 at pci0 dev 1 function 0 "Intel Core 4G PCIE" rev 0x06: msi >>> > pci1 at ppb0 bus 1 >>> > ix0 at pci1 dev 0 function 0 "Intel 82599" rev 0x01: msi, address >>> > 0c:c4:7a:59:30:a4 >>> > ix1 at pci1 dev 0 function 1 "Intel 82599" rev 0x01: msi, address >>> > 0c:c4:7a:59:30:a5 >>> > ppb1 at pci0 dev 1 function 1 "Intel Core 4G PCIE" rev 0x06: msi >>> > pci2 at ppb1 bus 2 >>> > ixl0 at pci2 dev 0 function 0 "Intel XL710 QSFP+" rev 0x02: port 0, FW >>> > 5.0.40043 API 1.5, msi, address 0c:c4:7a:5e:f9:c8 >>> > ixl0: unable to query phy types >>> > ixl1 at pci2 dev 0 function 1 "Intel XL710 QSFP+" rev 0x02: port 1, FW >>> > 5.0.40043 API 1.5, msi, address 0c:c4:7a:5e:f9:c9 >>> > ixl1: unable to query phy types >>> > "Intel 8 Series MEI" rev 0x04 at pci0 dev 22 function 0 not configured >>> > "Intel 8 Series MEI" rev 0x04 at pci0 dev 22 function 1 not configured >>> > ehci0 at pci0 dev 26 function 0 "Intel 8 Series USB" rev 0x05: apic 8 >>> int 16 >>> > usb0 at ehci0: USB revision 2.0 >>> > uhub0 at usb0 configuration 1 interface 0 "Intel EHCI root hub" rev >>> > 2.00/1.00 addr 1 >>> > ppb2 at pci0 dev 28 function 0 "Intel 8 Series PCIE" rev 0xd5: msi >>> > pci3 at ppb2 bus 3 >>> > ppb3 at pci3 dev 0 function 0 "ASPEED Technology AST1150 PCI" rev 0x03 >>> > pci4 at ppb3 bus 4 >>> > vga1 at pci4 dev 0 function 0 "ASPEED Technology AST2000" rev 0x30 >>> > wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) >>> > wsdisplay0: screen 1-5 added (80x25, vt100 emulation) >>> > ehci1 at pci0 dev 29 function 0 "Intel 8 Series USB" rev 0x05: apic 8 >>> int 22 >>> > usb1 at ehci1: USB revision 2.0 >>> > uhub1 at usb1 configuration 1 interface 0 "Intel EHCI root hub" rev >>> > 2.00/1.00 addr 1 >>> > pcib0 at pci0 dev 31 function 0 "Intel C224 LPC" rev 0x05 >>> > ahci0 at pci0 dev 31 function 2 "Intel 8 Series AHCI" rev 0x05: msi, >>> AHCI >>> > 1.3 >>> > ahci0: port 0: 6.0Gb/s >>> > ahci0: port 1: 6.0Gb/s >>> > scsibus1 at ahci0: 32 targets >>> > sd0 at scsibus1 targ 0 lun 0: <ATA, INTEL SSDSC2BB12, G201> SCSI3 >>> 0/direct >>> > fixed naa.55cd2e404c022db4 >>> > sd0: 114473MB, 512 bytes/sector, 234441648 sectors, thin >>> > sd1 at scsibus1 targ 1 lun 0: <ATA, INTEL SSDSC2BB12, G201> SCSI3 >>> 0/direct >>> > fixed naa.55cd2e404c022518 >>> > sd1: 114473MB, 512 bytes/sector, 234441648 sectors, thin >>> > ichiic0 at pci0 dev 31 function 3 "Intel 8 Series SMBus" rev 0x05: >>> apic 8 >>> > int 18 >>> > iic0 at ichiic0 >>> > sdtemp0 at iic0 addr 0x18: stts2002 >>> > sdtemp1 at iic0 addr 0x19: stts2002 >>> > sdtemp2 at iic0 addr 0x1a: stts2002 >>> > sdtemp3 at iic0 addr 0x1b: stts2002 >>> > spdmem0 at iic0 addr 0x50: 8GB DDR3 SDRAM ECC PC3-12800 with thermal >>> sensor >>> > spdmem1 at iic0 addr 0x51: 8GB DDR3 SDRAM ECC PC3-12800 with thermal >>> sensor >>> > spdmem2 at iic0 addr 0x52: 8GB DDR3 SDRAM ECC PC3-12800 with thermal >>> sensor >>> > spdmem3 at iic0 addr 0x53: 8GB DDR3 SDRAM ECC PC3-12800 with thermal >>> sensor >>> > "Intel 8 Series Thermal" rev 0x05 at pci0 dev 31 function 6 not >>> configured >>> > isa0 at pcib0 >>> > isadma0 at isa0 >>> > com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo >>> > com2 at isa0 port 0x3e8/8 irq 5: ns16550a, 16 byte fifo >>> > pckbc0 at isa0 port 0x60/5 irq 1 irq 12 >>> > pcppi0 at isa0 port 0x61 >>> > spkr0 at pcppi0 >>> > wbsio0 at isa0 port 0x2e/2: NCT6776F rev 0x33 >>> > lm1 at wbsio0 port 0xa30/8: NCT6776F >>> > vmm0 at mainbus0: VMX/EPT >>> > uhub2 at uhub0 port 1 configuration 1 interface 0 "Intel Rate Matching >>> Hub" >>> > rev 2.00/0.05 addr 2 >>> > uhub3 at uhub2 port 3 configuration 1 interface 0 "ATEN International >>> > product 0x7000" rev 2.00/0.00 addr 3 >>> > uhidev0 at uhub3 port 1 configuration 1 interface 0 "ATEN International >>> > product 0x2419" rev 1.10/1.00 addr 4 >>> > uhidev0: iclass 3/1 >>> > ukbd0 at uhidev0: 8 variable keys, 6 key codes >>> > wskbd0 at ukbd0: console keyboard, using wsdisplay0 >>> > uhidev1 at uhub3 port 1 configuration 1 interface 1 "ATEN International >>> > product 0x2419" rev 1.10/1.00 addr 4 >>> > uhidev1: iclass 3/1 >>> > ums0 at uhidev1: 3 buttons, Z dir >>> > wsmouse0 at ums0 mux 0 >>> > uhub4 at uhub1 port 1 configuration 1 interface 0 "Intel Rate Matching >>> Hub" >>> > rev 2.00/0.05 addr 2 >>> > vscsi0 at root >>> > scsibus2 at vscsi0: 256 targets >>> > softraid0 at root >>> > scsibus3 at softraid0: 256 targets >>> > root on sd0a (705899aa0befef4d.a) swap on sd0b dump on sd0b >>> >> >> >> I have a firewall test setup consisting of 14 machines. >> All old E3's, with a mix of ix and em, and some 2x40G ixl(work in >> progress). >> >> Firewalls interfaces are configured as: >> 2x ix => trunk(lacp) => vlans+carps >> >> 14 vlan interfaces and 13 carps (no carp on pfsync vlan interface) >> set skip on lo and the vlan carrying pfsync. >> No shaping or other funkiness, just pass/block. >> >> As this is a forwarding performance test with very low session rate >> the rule-set is irrelevant. >> >> The test performed is ping-pong of UDP packets across the firewall >> at the speed of RTT, additional clients are added to increase load. >> >> Testing with OpenBSD 6.5-beta (GENERIC.MP) #764: Sun Mar 3 10:24:08 MST >> 2019 >> Baseline, single session between testers. >> OpenBSD 6.5-beta (GENERIC.MP) #764: Sun Mar 3 10:24:08 MST 2019 >> >> thehoff@cloud5.swe69$ route -T2 exec ./phantom -k 1234 -c 25000 >> 10.96.2.21:2222 >> packet size: 512 >> packets per client: 25000 >> number of clients: 1 >> server address: 10.96.2.21:2222 >> pps: 4016 total drops: 0 avg rtt: 0.249 ms clients: 1 >> pps: 4015 total drops: 0 avg rtt: 0.249 ms clients: 1 >> pps: 3905 total drops: 0 avg rtt: 0.256 ms clients: 1 >> pps: 4016 total drops: 0 avg rtt: 0.249 ms clients: 1 >> pps: 4015 total drops: 0 avg rtt: 0.249 ms clients: 1 >> pps: 3976 total drops: 0 avg rtt: 0.252 ms clients: 1 >> Runtime: 7.118854881s Packets received: 25000 Packets dropped: 0 >> thehoff@cloud5.swe69$ >> >> The pps here is packets sent (and received), so firewall pps >> on trunk is double the above number. >> >> A ramp-up test where one client per second is added until packet loss: >> >> thehoff@cloud5.swe69$ route -T2 exec ./phantom -k 1234 -r 1 >> 10.96.2.21:2222 >> packet size: 512 >> ramp-up interval: 1 seconds >> packets per client: 9223372036854775807 >> number of clients: 9223372036854775807 >> server address: 10.96.2.21:2222 >> pps: 4051 total drops: 0 avg rtt: 0.494 ms clients: 2 >> pps: 5736 total drops: 0 avg rtt: 0.523 ms clients: 3 >> pps: 8258 total drops: 0 avg rtt: 0.484 ms clients: 4 >> pps: 10852 total drops: 0 avg rtt: 0.461 ms clients: 5 >> pps: 13174 total drops: 0 avg rtt: 0.455 ms clients: 6 >> pps: 15417 total drops: 0 avg rtt: 0.454 ms clients: 7 >> pps: 17894 total drops: 0 avg rtt: 0.447 ms clients: 8 >> pps: 20010 total drops: 0 avg rtt: 0.450 ms clients: 9 >> pps: 22178 total drops: 0 avg rtt: 0.451 ms clients: 10 >> pps: 24252 total drops: 0 avg rtt: 0.454 ms clients: 11 >> pps: 26056 total drops: 0 avg rtt: 0.461 ms clients: 12 >> pps: 28060 total drops: 0 avg rtt: 0.463 ms clients: 13 >> pps: 30437 total drops: 0 avg rtt: 0.460 ms clients: 14 >> pps: 32503 total drops: 0 avg rtt: 0.461 ms clients: 15 >> pps: 33421 total drops: 0 avg rtt: 0.479 ms clients: 16 >> pps: 35810 total drops: 0 avg rtt: 0.475 ms clients: 17 >> pps: 37042 total drops: 0 avg rtt: 0.486 ms clients: 18 >> pps: 38626 total drops: 0 avg rtt: 0.492 ms clients: 19 >> read failed: read udp 10.96.3.52:43580->10.96.2.21:2222: i/o timeout >> pps: 38045 total drops: 1 avg rtt: 0.526 ms clients: 20 >> read failed: read udp 10.96.3.52:3459->10.96.2.21:2222: i/o timeout >> >> Runtime: 20.198108441s Packets received: 472256 Packets dropped: 2 >> thehoff@cloud5.swe69$ >> >> Packet loss starts at around 35-40kpps: >> thehoff@cloud5.swe69$ route -T2 exec ./phantom -k 1234 -c 25000 -n 17 >> 10.96.2.21:2222 >> packet size: 512 >> packets per client: 25000 >> number of clients: 17 >> server address: 10.96.2.21:2222 >> pps: 35234 total drops: 0 avg rtt: 0.482 ms clients: 17 >> pps: 37294 total drops: 0 avg rtt: 0.456 ms clients: 17 >> pps: 36400 total drops: 0 avg rtt: 0.467 ms clients: 17 >> pps: 37040 total drops: 0 avg rtt: 0.459 ms clients: 17 >> pps: 37081 total drops: 0 avg rtt: 0.458 ms clients: 17 >> pps: 35258 total drops: 0 avg rtt: 0.482 ms clients: 17 >> read failed: read udp 10.96.3.52:45255->10.96.2.21:2222: i/o timeout >> pps: 36918 total drops: 1 avg rtt: 0.460 ms clients: 17 >> ^C >> Runtime: 8.029600463s Packets received: 285855 Packets dropped: 1 >> thehoff@cloud5.swe69$ >> >> >> Now testing with OpenBSD 6.5-beta (GENERIC.MP) #760: Thu Feb 28 12:38:40 >> MST 2019. >> Single-session base line: >> >> thehoff@cloud5.swe69$ route -T2 exec ./phantom -k 1234 -c 25000 >> 10.96.2.21:2222 >> packet size: 512 >> packets per client: 25000 >> number of clients: 1 >> server address: 10.96.2.21:2222 >> pps: 3640 total drops: 0 avg rtt: 0.275 ms clients: 1 >> pps: 3568 total drops: 0 avg rtt: 0.280 ms clients: 1 >> pps: 3638 total drops: 0 avg rtt: 0.275 ms clients: 1 >> pps: 3637 total drops: 0 avg rtt: 0.275 ms clients: 1 >> pps: 3641 total drops: 0 avg rtt: 0.275 ms clients: 1 >> pps: 3674 total drops: 0 avg rtt: 0.272 ms clients: 1 >> Runtime: 7.131570891s Packets received: 25000 Packets dropped: 0 >> thehoff@cloud5.swe69$ >> >> Ramp-up test: >> >> thehoff@cloud5.swe69$ route -T2 exec ./phantom -k 1234 -r 1 >> 10.96.2.21:2222 >> >> >> >> [2/1836] >> packet size: 512 >> ramp-up interval: 1 seconds >> packets per client: 9223372036854775807 >> number of clients: 9223372036854775807 >> server address: 10.96.2.21:2222 >> pps: 3827 total drops: 0 avg rtt: 0.523 ms clients: 2 >> pps: 5686 total drops: 0 avg rtt: 0.528 ms clients: 3 >> pps: 8278 total drops: 0 avg rtt: 0.483 ms clients: 4 >> pps: 10807 total drops: 0 avg rtt: 0.463 ms clients: 5 >> pps: 13402 total drops: 0 avg rtt: 0.448 ms clients: 6 >> pps: 15878 total drops: 0 avg rtt: 0.441 ms clients: 7 >> pps: 18432 total drops: 0 avg rtt: 0.434 ms clients: 8 >> pps: 20948 total drops: 0 avg rtt: 0.430 ms clients: 9 >> pps: 23393 total drops: 0 avg rtt: 0.427 ms clients: 10 >> pps: 25905 total drops: 0 avg rtt: 0.425 ms clients: 11 >> pps: 28186 total drops: 0 avg rtt: 0.426 ms clients: 12 >> pps: 29716 total drops: 0 avg rtt: 0.437 ms clients: 13 >> pps: 31801 total drops: 0 avg rtt: 0.440 ms clients: 14 >> pps: 32901 total drops: 0 avg rtt: 0.456 ms clients: 15 >> pps: 34493 total drops: 0 avg rtt: 0.464 ms clients: 16 >> pps: 35223 total drops: 0 avg rtt: 0.483 ms clients: 17 >> pps: 36824 total drops: 0 avg rtt: 0.489 ms clients: 18 >> pps: 38655 total drops: 0 avg rtt: 0.492 ms clients: 19 >> pps: 39625 total drops: 0 avg rtt: 0.505 ms clients: 20 >> pps: 41467 total drops: 0 avg rtt: 0.506 ms clients: 21 >> pps: 43342 total drops: 0 avg rtt: 0.508 ms clients: 22 >> pps: 44565 total drops: 0 avg rtt: 0.516 ms clients: 23 >> pps: 47022 total drops: 0 avg rtt: 0.510 ms clients: 24 >> pps: 48825 total drops: 0 avg rtt: 0.512 ms clients: 25 >> pps: 49426 total drops: 0 avg rtt: 0.526 ms clients: 26 >> pps: 51010 total drops: 0 avg rtt: 0.529 ms clients: 27 >> pps: 53227 total drops: 0 avg rtt: 0.526 ms clients: 28 >> pps: 55427 total drops: 0 avg rtt: 0.523 ms clients: 29 >> pps: 56294 total drops: 0 avg rtt: 0.533 ms clients: 30 >> pps: 57202 total drops: 0 avg rtt: 0.542 ms clients: 31 >> pps: 58672 total drops: 0 avg rtt: 0.545 ms clients: 32 >> pps: 60757 total drops: 0 avg rtt: 0.543 ms clients: 33 >> pps: 61619 total drops: 0 avg rtt: 0.552 ms clients: 34 >> pps: 62094 total drops: 0 avg rtt: 0.564 ms clients: 35 >> pps: 65039 total drops: 0 avg rtt: 0.554 ms clients: 36 >> pps: 65980 total drops: 0 avg rtt: 0.561 ms clients: 37 >> pps: 68124 total drops: 0 avg rtt: 0.558 ms clients: 38 >> pps: 68712 total drops: 0 avg rtt: 0.568 ms clients: 39 >> pps: 71362 total drops: 0 avg rtt: 0.561 ms clients: 40 >> pps: 72715 total drops: 0 avg rtt: 0.564 ms clients: 41 >> pps: 72362 total drops: 0 avg rtt: 0.580 ms clients: 42 >> pps: 72946 total drops: 0 avg rtt: 0.589 ms clients: 43 >> pps: 77915 total drops: 0 avg rtt: 0.565 ms clients: 44 >> pps: 77999 total drops: 0 avg rtt: 0.577 ms clients: 45 >> pps: 78544 total drops: 0 avg rtt: 0.586 ms clients: 46 >> pps: 80823 total drops: 0 avg rtt: 0.582 ms clients: 47 >> pps: 82208 total drops: 0 avg rtt: 0.584 ms clients: 48 >> pps: 83035 total drops: 0 avg rtt: 0.590 ms clients: 49 >> pps: 84476 total drops: 0 avg rtt: 0.592 ms clients: 50 >> pps: 86276 total drops: 0 avg rtt: 0.591 ms clients: 51 >> pps: 86409 total drops: 0 avg rtt: 0.602 ms clients: 52 >> pps: 87482 total drops: 0 avg rtt: 0.606 ms clients: 53 >> pps: 88760 total drops: 0 avg rtt: 0.608 ms clients: 54 >> pps: 90796 total drops: 0 avg rtt: 0.606 ms clients: 55 >> pps: 92476 total drops: 0 avg rtt: 0.606 ms clients: 56 >> pps: 94324 total drops: 0 avg rtt: 0.604 ms clients: 57 >> pps: 96194 total drops: 0 avg rtt: 0.603 ms clients: 58 >> pps: 98696 total drops: 0 avg rtt: 0.598 ms clients: 59 >> pps: 99291 total drops: 0 avg rtt: 0.604 ms clients: 60 >> pps: 100693 total drops: 0 avg rtt: 0.606 ms clients: 61 >> pps: 103156 total drops: 0 avg rtt: 0.601 ms clients: 62 >> pps: 104484 total drops: 0 avg rtt: 0.603 ms clients: 63 >> pps: 103819 total drops: 0 avg rtt: 0.616 ms clients: 64 >> pps: 105292 total drops: 0 avg rtt: 0.617 ms clients: 65 >> pps: 107878 total drops: 0 avg rtt: 0.612 ms clients: 66 >> pps: 108857 total drops: 0 avg rtt: 0.615 ms clients: 67 >> pps: 108004 total drops: 0 avg rtt: 0.630 ms clients: 68 >> pps: 108584 total drops: 0 avg rtt: 0.635 ms clients: 69 >> pps: 108634 total drops: 0 avg rtt: 0.644 ms clients: 70 >> pps: 109416 total drops: 0 avg rtt: 0.649 ms clients: 71 >> pps: 111989 total drops: 0 avg rtt: 0.643 ms clients: 72 >> pps: 112612 total drops: 0 avg rtt: 0.648 ms clients: 73 >> pps: 113058 total drops: 0 avg rtt: 0.655 ms clients: 74 >> pps: 114198 total drops: 0 avg rtt: 0.657 ms clients: 75 >> pps: 116640 total drops: 0 avg rtt: 0.652 ms clients: 76 >> pps: 117816 total drops: 0 avg rtt: 0.654 ms clients: 77 >> pps: 117980 total drops: 0 avg rtt: 0.661 ms clients: 78 >> pps: 120047 total drops: 0 avg rtt: 0.658 ms clients: 79 >> pps: 123420 total drops: 0 avg rtt: 0.648 ms clients: 80 >> pps: 123739 total drops: 0 avg rtt: 0.655 ms clients: 81 >> pps: 125930 total drops: 0 avg rtt: 0.651 ms clients: 82 >> pps: 127521 total drops: 0 avg rtt: 0.651 ms clients: 83 >> pps: 128499 total drops: 0 avg rtt: 0.654 ms clients: 84 >> pps: 129330 total drops: 0 avg rtt: 0.657 ms clients: 85 >> pps: 131785 total drops: 0 avg rtt: 0.653 ms clients: 86 >> pps: 131340 total drops: 0 avg rtt: 0.662 ms clients: 87 >> pps: 131660 total drops: 0 avg rtt: 0.668 ms clients: 88 >> pps: 133302 total drops: 0 avg rtt: 0.668 ms clients: 89 >> pps: 131994 total drops: 0 avg rtt: 0.682 ms clients: 90 >> pps: 133166 total drops: 0 avg rtt: 0.683 ms clients: 91 >> pps: 132691 total drops: 0 avg rtt: 0.693 ms clients: 92 >> pps: 132280 total drops: 0 avg rtt: 0.703 ms clients: 93 >> pps: 131950 total drops: 0 avg rtt: 0.712 ms clients: 94 >> pps: 133879 total drops: 0 avg rtt: 0.710 ms clients: 95 >> pps: 132591 total drops: 0 avg rtt: 0.724 ms clients: 96 >> pps: 131854 total drops: 0 avg rtt: 0.736 ms clients: 97 >> pps: 132559 total drops: 0 avg rtt: 0.739 ms clients: 98 >> pps: 132294 total drops: 0 avg rtt: 0.748 ms clients: 99 >> pps: 132859 total drops: 0 avg rtt: 0.753 ms clients: 100 >> pps: 131105 total drops: 0 avg rtt: 0.770 ms clients: 101 >> ^C >> Runtime: 1m41.78992826s Packets received: 8284037 Packets dropped: 0 >> thehoff@cloud5.swe69$ >> >> We are here beyond the point where the tester cpu's can cause false >> positives. >> >> With the older snapshot I can run 4 concurrent 30 client tests without >> packet >> loss, this is more than 300kpps on trunk0 on the firewall. >> Output from one of the four testers: >> >> thehoff@cloud5.swe69$ route -T2 exec ./phantom -k 1234 -c 0 -n 30 >> 10.96.2.21:2222 >> packet size: 512 >> packets per client: 9223372036854775807 >> number of clients: 30 >> server address: 10.96.2.21:2222 >> pps: 36448 total drops: 0 avg rtt: 0.823 ms clients: 30 >> pps: 40931 total drops: 0 avg rtt: 0.733 ms clients: 30 >> pps: 41419 total drops: 0 avg rtt: 0.724 ms clients: 30 >> pps: 41585 total drops: 0 avg rtt: 0.721 ms clients: 30 >> pps: 41078 total drops: 0 avg rtt: 0.730 ms clients: 30 >> pps: 41637 total drops: 0 avg rtt: 0.721 ms clients: 30 >> pps: 41504 total drops: 0 avg rtt: 0.723 ms clients: 30 >> pps: 41666 total drops: 0 avg rtt: 0.720 ms clients: 30 >> pps: 41803 total drops: 0 avg rtt: 0.718 ms clients: 30 >> pps: 42074 total drops: 0 avg rtt: 0.713 ms clients: 30 >> pps: 40728 total drops: 0 avg rtt: 0.737 ms clients: 30 >> ^C >> Runtime: 11.877603832s Packets received: 475599 Packets dropped: 0 >> thehoff@cloud5.swe69$ >> >> The recent snapshot takes me from over 300kpps clean, to less than 40kpps >> clean. >> The test clients runs 6.3, as that release is a lot faster here than 6.4 >> or snaps. >> >> Regards Tony >> >> > On the older snapshot, four tcp sessions with tcpbench resulted in 350kpps > in this setup. > The current snapshot gets around 100kpps, fluctuating between 70 and 120 > kpps. > > Don't be fooled by the named "cloud" on the testers. They are supermicro > microcloud boxes, > not cloud-cloud VM's. > > Regards Tony > >
> The recent snapshot takes me from over 300kpps > clean, to less than 40kpps clean. My bad, less than 80kpps clean on trunk0.
