Den mån 4 mars 2019 kl 09:26 skrev Tony Sarendal <t...@polarcap.org>:
> Den sön 3 mars 2019 kl 21:35 skrev Theo de Raadt <dera...@openbsd.org>: > >> Tony, >> >> Are you out of your mind? You didn't provide even a rough hint about >> what your firewall configuration looks like. You recognize that's >> pathetic, right? >> >> > Earlier in the week I could run parallel ping-pong tests through my test >> > firewalls >> > at 300kpps without any packet loss. I updated to the latest snapshot >> today >> > and >> > start to see packet loss at around 80kpps. >> > >> > /T >> > >> > OpenBSD 6.5-beta (GENERIC.MP) #764: Sun Mar 3 10:24:08 MST 2019 >> > dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/ >> GENERIC.MP >> > real mem = 34300891136 (32711MB) >> > avail mem = 33251393536 (31711MB) >> > mpath0 at root >> > scsibus0 at mpath0: 256 targets >> > mainbus0 at root >> > bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xec170 (34 entries) >> > bios0: vendor American Megatrends Inc. version "3.0" date 04/24/2015 >> > bios0: Supermicro X10SLD >> > acpi0 at bios0: rev 2 >> > acpi0: sleep states S0 S4 S5 >> > acpi0: tables DSDT FACP APIC FPDT FIDT SSDT SSDT MCFG PRAD HPET SSDT >> SSDT >> > SPMI DMAR EINJ ERST HEST BERT >> > acpi0: wakeup devices PEGP(S4) PEG0(S4) PEGP(S4) PEG1(S4) PEGP(S4) >> PEG2(S4) >> > PXSX(S4) RP01(S4) PXSX(S4) RP02(S4) PXSX(S4) RP03(S4) PXSX(S4) RP04(S4) >> > PXSX(S4) RP05(S4) [...] >> > acpitimer0 at acpi0: 3579545 Hz, 24 bits >> > acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat >> > cpu0 at mainbus0: apid 0 (boot processor) >> > cpu0: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.68 MHz, 06-3c-03 >> > cpu0: >> > >> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN >> > cpu0: 256KB 64b/line 8-way L2 cache >> > cpu0: smt 0, core 0, package 0 >> > mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges >> > cpu0: apic clock running at 99MHz >> > cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4, IBE >> > cpu1 at mainbus0: apid 2 (application processor) >> > cpu1: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.01 MHz, 06-3c-03 >> > cpu1: >> > >> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN >> > cpu1: 256KB 64b/line 8-way L2 cache >> > cpu1: smt 0, core 1, package 0 >> > cpu2 at mainbus0: apid 4 (application processor) >> > cpu2: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.01 MHz, 06-3c-03 >> > cpu2: >> > >> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN >> > cpu2: 256KB 64b/line 8-way L2 cache >> > cpu2: smt 0, core 2, package 0 >> > cpu3 at mainbus0: apid 6 (application processor) >> > cpu3: Intel(R) Xeon(R) CPU E3-1241 v3 @ 3.50GHz, 3500.01 MHz, 06-3c-03 >> > cpu3: >> > >> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,PERF,ITSC,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN >> > cpu3: 256KB 64b/line 8-way L2 cache >> > cpu3: smt 0, core 3, package 0 >> > ioapic0 at mainbus0: apid 8 pa 0xfec00000, version 20, 24 pins >> > acpimcfg0 at acpi0 >> > acpimcfg0: addr 0xf8000000, bus 0-63 >> > acpihpet0 at acpi0: 14318179 Hz >> > acpiprt0 at acpi0: bus 0 (PCI0) >> > acpiprt1 at acpi0: bus 1 (PEG0) >> > acpiprt2 at acpi0: bus 2 (PEG1) >> > acpiprt3 at acpi0: bus -1 (PEG2) >> > acpiprt4 at acpi0: bus 3 (RP01) >> > acpiprt5 at acpi0: bus -1 (RP02) >> > acpiprt6 at acpi0: bus -1 (RP03) >> > acpiprt7 at acpi0: bus -1 (RP04) >> > acpiprt8 at acpi0: bus -1 (RP05) >> > acpiprt9 at acpi0: bus -1 (RP06) >> > acpiprt10 at acpi0: bus -1 (RP07) >> > acpiprt11 at acpi0: bus -1 (RP08) >> > acpiec0 at acpi0: not present >> > acpicpu0 at acpi0: C1(@1 halt!) >> > acpicpu1 at acpi0: C1(@1 halt!) >> > acpicpu2 at acpi0: C1(@1 halt!) >> > acpicpu3 at acpi0: C1(@1 halt!) >> > acpipwrres0 at acpi0: PG00, resource for PEG0 >> > acpipwrres1 at acpi0: PG01, resource for PEG1 >> > acpipwrres2 at acpi0: PG02, resource for PEG2 >> > acpipwrres3 at acpi0: FN00, resource for FAN0 >> > acpipwrres4 at acpi0: FN01, resource for FAN1 >> > acpipwrres5 at acpi0: FN02, resource for FAN2 >> > acpipwrres6 at acpi0: FN03, resource for FAN3 >> > acpipwrres7 at acpi0: FN04, resource for FAN4 >> > acpitz0 at acpi0: critical temperature is 105 degC >> > acpitz1 at acpi0: critical temperature is 105 degC >> > acpipci0 at acpi0 PCI0: 0x00000000 0x00000011 0x00000001 >> > acpicmos0 at acpi0 >> > "IPI0001" at acpi0 not configured >> > acpibtn0 at acpi0: SLPB >> > acpibtn1 at acpi0: PWRB >> > "PNP0C0B" at acpi0 not configured >> > "PNP0C0B" at acpi0 not configured >> > "PNP0C0B" at acpi0 not configured >> > "PNP0C0B" at acpi0 not configured >> > "PNP0C0B" at acpi0 not configured >> > acpivideo0 at acpi0: GFX0 >> > acpivout0 at acpivideo0: DD1F >> > ipmi at mainbus0 not configured >> > pci0 at mainbus0 bus 0 >> > pchb0 at pci0 dev 0 function 0 "Intel Xeon E3-1200 v3 Host" rev 0x06 >> > ppb0 at pci0 dev 1 function 0 "Intel Core 4G PCIE" rev 0x06: msi >> > pci1 at ppb0 bus 1 >> > ix0 at pci1 dev 0 function 0 "Intel 82599" rev 0x01: msi, address >> > 0c:c4:7a:59:30:a4 >> > ix1 at pci1 dev 0 function 1 "Intel 82599" rev 0x01: msi, address >> > 0c:c4:7a:59:30:a5 >> > ppb1 at pci0 dev 1 function 1 "Intel Core 4G PCIE" rev 0x06: msi >> > pci2 at ppb1 bus 2 >> > ixl0 at pci2 dev 0 function 0 "Intel XL710 QSFP+" rev 0x02: port 0, FW >> > 5.0.40043 API 1.5, msi, address 0c:c4:7a:5e:f9:c8 >> > ixl0: unable to query phy types >> > ixl1 at pci2 dev 0 function 1 "Intel XL710 QSFP+" rev 0x02: port 1, FW >> > 5.0.40043 API 1.5, msi, address 0c:c4:7a:5e:f9:c9 >> > ixl1: unable to query phy types >> > "Intel 8 Series MEI" rev 0x04 at pci0 dev 22 function 0 not configured >> > "Intel 8 Series MEI" rev 0x04 at pci0 dev 22 function 1 not configured >> > ehci0 at pci0 dev 26 function 0 "Intel 8 Series USB" rev 0x05: apic 8 >> int 16 >> > usb0 at ehci0: USB revision 2.0 >> > uhub0 at usb0 configuration 1 interface 0 "Intel EHCI root hub" rev >> > 2.00/1.00 addr 1 >> > ppb2 at pci0 dev 28 function 0 "Intel 8 Series PCIE" rev 0xd5: msi >> > pci3 at ppb2 bus 3 >> > ppb3 at pci3 dev 0 function 0 "ASPEED Technology AST1150 PCI" rev 0x03 >> > pci4 at ppb3 bus 4 >> > vga1 at pci4 dev 0 function 0 "ASPEED Technology AST2000" rev 0x30 >> > wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) >> > wsdisplay0: screen 1-5 added (80x25, vt100 emulation) >> > ehci1 at pci0 dev 29 function 0 "Intel 8 Series USB" rev 0x05: apic 8 >> int 22 >> > usb1 at ehci1: USB revision 2.0 >> > uhub1 at usb1 configuration 1 interface 0 "Intel EHCI root hub" rev >> > 2.00/1.00 addr 1 >> > pcib0 at pci0 dev 31 function 0 "Intel C224 LPC" rev 0x05 >> > ahci0 at pci0 dev 31 function 2 "Intel 8 Series AHCI" rev 0x05: msi, >> AHCI >> > 1.3 >> > ahci0: port 0: 6.0Gb/s >> > ahci0: port 1: 6.0Gb/s >> > scsibus1 at ahci0: 32 targets >> > sd0 at scsibus1 targ 0 lun 0: <ATA, INTEL SSDSC2BB12, G201> SCSI3 >> 0/direct >> > fixed naa.55cd2e404c022db4 >> > sd0: 114473MB, 512 bytes/sector, 234441648 sectors, thin >> > sd1 at scsibus1 targ 1 lun 0: <ATA, INTEL SSDSC2BB12, G201> SCSI3 >> 0/direct >> > fixed naa.55cd2e404c022518 >> > sd1: 114473MB, 512 bytes/sector, 234441648 sectors, thin >> > ichiic0 at pci0 dev 31 function 3 "Intel 8 Series SMBus" rev 0x05: apic >> 8 >> > int 18 >> > iic0 at ichiic0 >> > sdtemp0 at iic0 addr 0x18: stts2002 >> > sdtemp1 at iic0 addr 0x19: stts2002 >> > sdtemp2 at iic0 addr 0x1a: stts2002 >> > sdtemp3 at iic0 addr 0x1b: stts2002 >> > spdmem0 at iic0 addr 0x50: 8GB DDR3 SDRAM ECC PC3-12800 with thermal >> sensor >> > spdmem1 at iic0 addr 0x51: 8GB DDR3 SDRAM ECC PC3-12800 with thermal >> sensor >> > spdmem2 at iic0 addr 0x52: 8GB DDR3 SDRAM ECC PC3-12800 with thermal >> sensor >> > spdmem3 at iic0 addr 0x53: 8GB DDR3 SDRAM ECC PC3-12800 with thermal >> sensor >> > "Intel 8 Series Thermal" rev 0x05 at pci0 dev 31 function 6 not >> configured >> > isa0 at pcib0 >> > isadma0 at isa0 >> > com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo >> > com2 at isa0 port 0x3e8/8 irq 5: ns16550a, 16 byte fifo >> > pckbc0 at isa0 port 0x60/5 irq 1 irq 12 >> > pcppi0 at isa0 port 0x61 >> > spkr0 at pcppi0 >> > wbsio0 at isa0 port 0x2e/2: NCT6776F rev 0x33 >> > lm1 at wbsio0 port 0xa30/8: NCT6776F >> > vmm0 at mainbus0: VMX/EPT >> > uhub2 at uhub0 port 1 configuration 1 interface 0 "Intel Rate Matching >> Hub" >> > rev 2.00/0.05 addr 2 >> > uhub3 at uhub2 port 3 configuration 1 interface 0 "ATEN International >> > product 0x7000" rev 2.00/0.00 addr 3 >> > uhidev0 at uhub3 port 1 configuration 1 interface 0 "ATEN International >> > product 0x2419" rev 1.10/1.00 addr 4 >> > uhidev0: iclass 3/1 >> > ukbd0 at uhidev0: 8 variable keys, 6 key codes >> > wskbd0 at ukbd0: console keyboard, using wsdisplay0 >> > uhidev1 at uhub3 port 1 configuration 1 interface 1 "ATEN International >> > product 0x2419" rev 1.10/1.00 addr 4 >> > uhidev1: iclass 3/1 >> > ums0 at uhidev1: 3 buttons, Z dir >> > wsmouse0 at ums0 mux 0 >> > uhub4 at uhub1 port 1 configuration 1 interface 0 "Intel Rate Matching >> Hub" >> > rev 2.00/0.05 addr 2 >> > vscsi0 at root >> > scsibus2 at vscsi0: 256 targets >> > softraid0 at root >> > scsibus3 at softraid0: 256 targets >> > root on sd0a (705899aa0befef4d.a) swap on sd0b dump on sd0b >> > > > I have a firewall test setup consisting of 14 machines. > All old E3's, with a mix of ix and em, and some 2x40G ixl(work in > progress). > > Firewalls interfaces are configured as: > 2x ix => trunk(lacp) => vlans+carps > > 14 vlan interfaces and 13 carps (no carp on pfsync vlan interface) > set skip on lo and the vlan carrying pfsync. > No shaping or other funkiness, just pass/block. > > As this is a forwarding performance test with very low session rate > the rule-set is irrelevant. > > The test performed is ping-pong of UDP packets across the firewall > at the speed of RTT, additional clients are added to increase load. > > Testing with OpenBSD 6.5-beta (GENERIC.MP) #764: Sun Mar 3 10:24:08 MST > 2019 > Baseline, single session between testers. > OpenBSD 6.5-beta (GENERIC.MP) #764: Sun Mar 3 10:24:08 MST 2019 > > thehoff@cloud5.swe69$ route -T2 exec ./phantom -k 1234 -c 25000 > 10.96.2.21:2222 > packet size: 512 > packets per client: 25000 > number of clients: 1 > server address: 10.96.2.21:2222 > pps: 4016 total drops: 0 avg rtt: 0.249 ms clients: 1 > pps: 4015 total drops: 0 avg rtt: 0.249 ms clients: 1 > pps: 3905 total drops: 0 avg rtt: 0.256 ms clients: 1 > pps: 4016 total drops: 0 avg rtt: 0.249 ms clients: 1 > pps: 4015 total drops: 0 avg rtt: 0.249 ms clients: 1 > pps: 3976 total drops: 0 avg rtt: 0.252 ms clients: 1 > Runtime: 7.118854881s Packets received: 25000 Packets dropped: 0 > thehoff@cloud5.swe69$ > > The pps here is packets sent (and received), so firewall pps > on trunk is double the above number. > > A ramp-up test where one client per second is added until packet loss: > > thehoff@cloud5.swe69$ route -T2 exec ./phantom -k 1234 -r 1 > 10.96.2.21:2222 > packet size: 512 > ramp-up interval: 1 seconds > packets per client: 9223372036854775807 > number of clients: 9223372036854775807 > server address: 10.96.2.21:2222 > pps: 4051 total drops: 0 avg rtt: 0.494 ms clients: 2 > pps: 5736 total drops: 0 avg rtt: 0.523 ms clients: 3 > pps: 8258 total drops: 0 avg rtt: 0.484 ms clients: 4 > pps: 10852 total drops: 0 avg rtt: 0.461 ms clients: 5 > pps: 13174 total drops: 0 avg rtt: 0.455 ms clients: 6 > pps: 15417 total drops: 0 avg rtt: 0.454 ms clients: 7 > pps: 17894 total drops: 0 avg rtt: 0.447 ms clients: 8 > pps: 20010 total drops: 0 avg rtt: 0.450 ms clients: 9 > pps: 22178 total drops: 0 avg rtt: 0.451 ms clients: 10 > pps: 24252 total drops: 0 avg rtt: 0.454 ms clients: 11 > pps: 26056 total drops: 0 avg rtt: 0.461 ms clients: 12 > pps: 28060 total drops: 0 avg rtt: 0.463 ms clients: 13 > pps: 30437 total drops: 0 avg rtt: 0.460 ms clients: 14 > pps: 32503 total drops: 0 avg rtt: 0.461 ms clients: 15 > pps: 33421 total drops: 0 avg rtt: 0.479 ms clients: 16 > pps: 35810 total drops: 0 avg rtt: 0.475 ms clients: 17 > pps: 37042 total drops: 0 avg rtt: 0.486 ms clients: 18 > pps: 38626 total drops: 0 avg rtt: 0.492 ms clients: 19 > read failed: read udp 10.96.3.52:43580->10.96.2.21:2222: i/o timeout > pps: 38045 total drops: 1 avg rtt: 0.526 ms clients: 20 > read failed: read udp 10.96.3.52:3459->10.96.2.21:2222: i/o timeout > > Runtime: 20.198108441s Packets received: 472256 Packets dropped: 2 > thehoff@cloud5.swe69$ > > Packet loss starts at around 35-40kpps: > thehoff@cloud5.swe69$ route -T2 exec ./phantom -k 1234 -c 25000 -n 17 > 10.96.2.21:2222 > packet size: 512 > packets per client: 25000 > number of clients: 17 > server address: 10.96.2.21:2222 > pps: 35234 total drops: 0 avg rtt: 0.482 ms clients: 17 > pps: 37294 total drops: 0 avg rtt: 0.456 ms clients: 17 > pps: 36400 total drops: 0 avg rtt: 0.467 ms clients: 17 > pps: 37040 total drops: 0 avg rtt: 0.459 ms clients: 17 > pps: 37081 total drops: 0 avg rtt: 0.458 ms clients: 17 > pps: 35258 total drops: 0 avg rtt: 0.482 ms clients: 17 > read failed: read udp 10.96.3.52:45255->10.96.2.21:2222: i/o timeout > pps: 36918 total drops: 1 avg rtt: 0.460 ms clients: 17 > ^C > Runtime: 8.029600463s Packets received: 285855 Packets dropped: 1 > thehoff@cloud5.swe69$ > > > Now testing with OpenBSD 6.5-beta (GENERIC.MP) #760: Thu Feb 28 12:38:40 > MST 2019. > Single-session base line: > > thehoff@cloud5.swe69$ route -T2 exec ./phantom -k 1234 -c 25000 > 10.96.2.21:2222 > packet size: 512 > packets per client: 25000 > number of clients: 1 > server address: 10.96.2.21:2222 > pps: 3640 total drops: 0 avg rtt: 0.275 ms clients: 1 > pps: 3568 total drops: 0 avg rtt: 0.280 ms clients: 1 > pps: 3638 total drops: 0 avg rtt: 0.275 ms clients: 1 > pps: 3637 total drops: 0 avg rtt: 0.275 ms clients: 1 > pps: 3641 total drops: 0 avg rtt: 0.275 ms clients: 1 > pps: 3674 total drops: 0 avg rtt: 0.272 ms clients: 1 > Runtime: 7.131570891s Packets received: 25000 Packets dropped: 0 > thehoff@cloud5.swe69$ > > Ramp-up test: > > thehoff@cloud5.swe69$ route -T2 exec ./phantom -k 1234 -r 1 > 10.96.2.21:2222 > > > > [2/1836] > packet size: 512 > ramp-up interval: 1 seconds > packets per client: 9223372036854775807 > number of clients: 9223372036854775807 > server address: 10.96.2.21:2222 > pps: 3827 total drops: 0 avg rtt: 0.523 ms clients: 2 > pps: 5686 total drops: 0 avg rtt: 0.528 ms clients: 3 > pps: 8278 total drops: 0 avg rtt: 0.483 ms clients: 4 > pps: 10807 total drops: 0 avg rtt: 0.463 ms clients: 5 > pps: 13402 total drops: 0 avg rtt: 0.448 ms clients: 6 > pps: 15878 total drops: 0 avg rtt: 0.441 ms clients: 7 > pps: 18432 total drops: 0 avg rtt: 0.434 ms clients: 8 > pps: 20948 total drops: 0 avg rtt: 0.430 ms clients: 9 > pps: 23393 total drops: 0 avg rtt: 0.427 ms clients: 10 > pps: 25905 total drops: 0 avg rtt: 0.425 ms clients: 11 > pps: 28186 total drops: 0 avg rtt: 0.426 ms clients: 12 > pps: 29716 total drops: 0 avg rtt: 0.437 ms clients: 13 > pps: 31801 total drops: 0 avg rtt: 0.440 ms clients: 14 > pps: 32901 total drops: 0 avg rtt: 0.456 ms clients: 15 > pps: 34493 total drops: 0 avg rtt: 0.464 ms clients: 16 > pps: 35223 total drops: 0 avg rtt: 0.483 ms clients: 17 > pps: 36824 total drops: 0 avg rtt: 0.489 ms clients: 18 > pps: 38655 total drops: 0 avg rtt: 0.492 ms clients: 19 > pps: 39625 total drops: 0 avg rtt: 0.505 ms clients: 20 > pps: 41467 total drops: 0 avg rtt: 0.506 ms clients: 21 > pps: 43342 total drops: 0 avg rtt: 0.508 ms clients: 22 > pps: 44565 total drops: 0 avg rtt: 0.516 ms clients: 23 > pps: 47022 total drops: 0 avg rtt: 0.510 ms clients: 24 > pps: 48825 total drops: 0 avg rtt: 0.512 ms clients: 25 > pps: 49426 total drops: 0 avg rtt: 0.526 ms clients: 26 > pps: 51010 total drops: 0 avg rtt: 0.529 ms clients: 27 > pps: 53227 total drops: 0 avg rtt: 0.526 ms clients: 28 > pps: 55427 total drops: 0 avg rtt: 0.523 ms clients: 29 > pps: 56294 total drops: 0 avg rtt: 0.533 ms clients: 30 > pps: 57202 total drops: 0 avg rtt: 0.542 ms clients: 31 > pps: 58672 total drops: 0 avg rtt: 0.545 ms clients: 32 > pps: 60757 total drops: 0 avg rtt: 0.543 ms clients: 33 > pps: 61619 total drops: 0 avg rtt: 0.552 ms clients: 34 > pps: 62094 total drops: 0 avg rtt: 0.564 ms clients: 35 > pps: 65039 total drops: 0 avg rtt: 0.554 ms clients: 36 > pps: 65980 total drops: 0 avg rtt: 0.561 ms clients: 37 > pps: 68124 total drops: 0 avg rtt: 0.558 ms clients: 38 > pps: 68712 total drops: 0 avg rtt: 0.568 ms clients: 39 > pps: 71362 total drops: 0 avg rtt: 0.561 ms clients: 40 > pps: 72715 total drops: 0 avg rtt: 0.564 ms clients: 41 > pps: 72362 total drops: 0 avg rtt: 0.580 ms clients: 42 > pps: 72946 total drops: 0 avg rtt: 0.589 ms clients: 43 > pps: 77915 total drops: 0 avg rtt: 0.565 ms clients: 44 > pps: 77999 total drops: 0 avg rtt: 0.577 ms clients: 45 > pps: 78544 total drops: 0 avg rtt: 0.586 ms clients: 46 > pps: 80823 total drops: 0 avg rtt: 0.582 ms clients: 47 > pps: 82208 total drops: 0 avg rtt: 0.584 ms clients: 48 > pps: 83035 total drops: 0 avg rtt: 0.590 ms clients: 49 > pps: 84476 total drops: 0 avg rtt: 0.592 ms clients: 50 > pps: 86276 total drops: 0 avg rtt: 0.591 ms clients: 51 > pps: 86409 total drops: 0 avg rtt: 0.602 ms clients: 52 > pps: 87482 total drops: 0 avg rtt: 0.606 ms clients: 53 > pps: 88760 total drops: 0 avg rtt: 0.608 ms clients: 54 > pps: 90796 total drops: 0 avg rtt: 0.606 ms clients: 55 > pps: 92476 total drops: 0 avg rtt: 0.606 ms clients: 56 > pps: 94324 total drops: 0 avg rtt: 0.604 ms clients: 57 > pps: 96194 total drops: 0 avg rtt: 0.603 ms clients: 58 > pps: 98696 total drops: 0 avg rtt: 0.598 ms clients: 59 > pps: 99291 total drops: 0 avg rtt: 0.604 ms clients: 60 > pps: 100693 total drops: 0 avg rtt: 0.606 ms clients: 61 > pps: 103156 total drops: 0 avg rtt: 0.601 ms clients: 62 > pps: 104484 total drops: 0 avg rtt: 0.603 ms clients: 63 > pps: 103819 total drops: 0 avg rtt: 0.616 ms clients: 64 > pps: 105292 total drops: 0 avg rtt: 0.617 ms clients: 65 > pps: 107878 total drops: 0 avg rtt: 0.612 ms clients: 66 > pps: 108857 total drops: 0 avg rtt: 0.615 ms clients: 67 > pps: 108004 total drops: 0 avg rtt: 0.630 ms clients: 68 > pps: 108584 total drops: 0 avg rtt: 0.635 ms clients: 69 > pps: 108634 total drops: 0 avg rtt: 0.644 ms clients: 70 > pps: 109416 total drops: 0 avg rtt: 0.649 ms clients: 71 > pps: 111989 total drops: 0 avg rtt: 0.643 ms clients: 72 > pps: 112612 total drops: 0 avg rtt: 0.648 ms clients: 73 > pps: 113058 total drops: 0 avg rtt: 0.655 ms clients: 74 > pps: 114198 total drops: 0 avg rtt: 0.657 ms clients: 75 > pps: 116640 total drops: 0 avg rtt: 0.652 ms clients: 76 > pps: 117816 total drops: 0 avg rtt: 0.654 ms clients: 77 > pps: 117980 total drops: 0 avg rtt: 0.661 ms clients: 78 > pps: 120047 total drops: 0 avg rtt: 0.658 ms clients: 79 > pps: 123420 total drops: 0 avg rtt: 0.648 ms clients: 80 > pps: 123739 total drops: 0 avg rtt: 0.655 ms clients: 81 > pps: 125930 total drops: 0 avg rtt: 0.651 ms clients: 82 > pps: 127521 total drops: 0 avg rtt: 0.651 ms clients: 83 > pps: 128499 total drops: 0 avg rtt: 0.654 ms clients: 84 > pps: 129330 total drops: 0 avg rtt: 0.657 ms clients: 85 > pps: 131785 total drops: 0 avg rtt: 0.653 ms clients: 86 > pps: 131340 total drops: 0 avg rtt: 0.662 ms clients: 87 > pps: 131660 total drops: 0 avg rtt: 0.668 ms clients: 88 > pps: 133302 total drops: 0 avg rtt: 0.668 ms clients: 89 > pps: 131994 total drops: 0 avg rtt: 0.682 ms clients: 90 > pps: 133166 total drops: 0 avg rtt: 0.683 ms clients: 91 > pps: 132691 total drops: 0 avg rtt: 0.693 ms clients: 92 > pps: 132280 total drops: 0 avg rtt: 0.703 ms clients: 93 > pps: 131950 total drops: 0 avg rtt: 0.712 ms clients: 94 > pps: 133879 total drops: 0 avg rtt: 0.710 ms clients: 95 > pps: 132591 total drops: 0 avg rtt: 0.724 ms clients: 96 > pps: 131854 total drops: 0 avg rtt: 0.736 ms clients: 97 > pps: 132559 total drops: 0 avg rtt: 0.739 ms clients: 98 > pps: 132294 total drops: 0 avg rtt: 0.748 ms clients: 99 > pps: 132859 total drops: 0 avg rtt: 0.753 ms clients: 100 > pps: 131105 total drops: 0 avg rtt: 0.770 ms clients: 101 > ^C > Runtime: 1m41.78992826s Packets received: 8284037 Packets dropped: 0 > thehoff@cloud5.swe69$ > > We are here beyond the point where the tester cpu's can cause false > positives. > > With the older snapshot I can run 4 concurrent 30 client tests without > packet > loss, this is more than 300kpps on trunk0 on the firewall. > Output from one of the four testers: > > thehoff@cloud5.swe69$ route -T2 exec ./phantom -k 1234 -c 0 -n 30 > 10.96.2.21:2222 > packet size: 512 > packets per client: 9223372036854775807 > number of clients: 30 > server address: 10.96.2.21:2222 > pps: 36448 total drops: 0 avg rtt: 0.823 ms clients: 30 > pps: 40931 total drops: 0 avg rtt: 0.733 ms clients: 30 > pps: 41419 total drops: 0 avg rtt: 0.724 ms clients: 30 > pps: 41585 total drops: 0 avg rtt: 0.721 ms clients: 30 > pps: 41078 total drops: 0 avg rtt: 0.730 ms clients: 30 > pps: 41637 total drops: 0 avg rtt: 0.721 ms clients: 30 > pps: 41504 total drops: 0 avg rtt: 0.723 ms clients: 30 > pps: 41666 total drops: 0 avg rtt: 0.720 ms clients: 30 > pps: 41803 total drops: 0 avg rtt: 0.718 ms clients: 30 > pps: 42074 total drops: 0 avg rtt: 0.713 ms clients: 30 > pps: 40728 total drops: 0 avg rtt: 0.737 ms clients: 30 > ^C > Runtime: 11.877603832s Packets received: 475599 Packets dropped: 0 > thehoff@cloud5.swe69$ > > The recent snapshot takes me from over 300kpps clean, to less than 40kpps > clean. > The test clients runs 6.3, as that release is a lot faster here than 6.4 > or snaps. > > Regards Tony > > On the older snapshot, four tcp sessions with tcpbench resulted in 350kpps in this setup. The current snapshot gets around 100kpps, fluctuating between 70 and 120 kpps. Don't be fooled by the named "cloud" on the testers. They are supermicro microcloud boxes, not cloud-cloud VM's. Regards Tony
