Hello! I setup StrongSwan and I am currently performing some throughput tests.
I have two VM-Ware ESXi servers and on each of these is a VM running Ubuntu 16.04. Each VM has 8 virtual CPUs. The VM-Ware servers are interconnected with an 10G link. The performance on the link is about ~500 MBytes/s. This is because there currently other productive instances running in parallel. Without these I will get about ~950 MBytes/s. But this should be enough for my tests... Now my problem: If I run the IPSec connection without the pcrypt module loaded, I will get the following throughput: > ./netperf -H 192.168.184.250 -fM -D 5 -c 1 -C 1 -l 30 Recv Send Send Utilization Service Demand Socket Socket Message Elapsed Send Recv Send Recv Size Size Size Time Throughput local remote local remote bytes bytes bytes secs. MBytes /s % S % S us/KB us/KB 87380 16384 16384 30.01 152.25 10.52 6.21 5.397 3.186 87380 16384 16384 30.00 157.79 11.05 7.82 5.469 3.872 87380 16384 16384 30.01 167.69 11.94 6.79 5.560 3.162 87380 16384 16384 30.00 165.32 11.82 7.19 5.585 3.398 If I run this with the pcrypt module loaded, I will get the following throughput: > ./netperf -H 192.168.184.250 -fM -D 5 -c 1 -C 1 -l 30 Recv Send Send Utilization Service Demand Socket Socket Message Elapsed Send Recv Send Recv Size Size Size Time Throughput local remote local remote bytes bytes bytes secs. MBytes /s % S % S us/KB us/KB 87380 16384 16384 30.02 93.67 21.12 17.45 17.615 14.557 87380 16384 16384 30.02 87.30 21.81 17.14 19.515 15.338 87380 16384 16384 30.02 90.50 21.23 4.29 18.324 3.706 87380 16384 16384 30.02 92.18 20.97 4.19 17.771 3.548 [Beware: All throughput values are in MBytes/s not Mbits/s !] Can anybody explain this? I expected the throughput to increase not to decrease... I loaded the pcrypt module with: modprobe tcrypt alg="pcrypt(rfc4106(gcm(aes)))" type=3 (and restarted strongswan with "systemctrl restart strongswan") I played with the "replay_window" option (tried 0 and 64), but it did not help. What could cause this slowdown? Any kind of help is appreciated... Regards Sven This is my configuration: ------------------------- # /etc/ipsec.conf - strongSwan IPsec configuration file config setup conn %default ikelifetime=60m keylife=20m rekeymargin=3m keyingtries=1 keyexchange=ikev2 mobike=no esp=aes256gcm16! conn net-net left=10.10.99.1 leftcert=moonCert.pem leftid=@moon.strongswan.org leftsubnet=192.168.183.0/24 leftfirewall=yes right=10.10.99.2 rightid=@sun.strongswan.org rightsubnet=192.168.184.0/24 auto=add replay_window=0 (Same on the other side, just the IPs swapped and with the sunCert.pem certificate ...) My strongswan output: charon[4392]: 00[DMN] Starting IKE charon daemon (strongSwan 5.3.5, Linux 4.4.0-93-generic, x86_64) charon[4392]: 00[CFG] loading ca certificates from '/etc/ipsec.d/cacerts' charon[4392]: 00[CFG] loaded ca certificate "C=CH, O=Linux strongSwan, CN=strongSwan Root CA" from '/etc/ipsec.d/cacerts/strongswanCert.pem' charon[4392]: 00[CFG] loading aa certificates from '/etc/ipsec.d/aacerts' charon[4392]: 00[CFG] loading ocsp signer certificates from '/etc/ipsec.d/ocspcerts' charon[4392]: 00[CFG] loading attribute certificates from '/etc/ipsec.d/acerts' charon[4392]: 00[CFG] loading crls from '/etc/ipsec.d/crls' charon[4392]: 00[CFG] loading secrets from '/etc/ipsec.secrets' charon[4392]: 00[CFG] loaded RSA private key from '/etc/ipsec.d/private/moonKey.pem' charon[4392]: 00[CFG] loaded IKE secret for @moon.strongswan.org @sun.strongswan.org charon[4392]: 00[CFG] opening triplet file /etc/ipsec.d/triplets.dat failed: No such file or directory charon[4392]: 00[CFG] loaded 0 RADIUS server configurations charon[4392]: 00[LIB] loaded plugins: charon test-vectors aes rc2 sha1 sha2 md4 md5 random nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs8 pkcs12 pgp dnskey sshkey pem openssl fips-prf gmp agent xcbc hmac gcm attr kernel-netlink resolve socket-default connmark farp stroke updown eap-identity eap-sim eap-sim-pcsc eap-aka eap-aka-3gpp2 eap-simaka-pseudonym eap-simaka-reauth eap-md5 eap-gtc eap-mschapv2 eap-dynamic eap-radius eap-tls eap-ttls eap-peap eap-tnc xauth-generic xauth-eap xauth-pam xauth-noauth tnc-tnccs tnccs-20 tnccs-11 tnccs-dynamic dhcp lookip error-notify certexpire led addrblock unity charon[4392]: 00[LIB] dropped capabilities, running as uid 0, gid 0 charon[4392]: 00[JOB] spawning 16 worker threads ipsec_starter[4391]: charon (4392) started after 40 ms charon[4392]: 06[CFG] received stroke: add connection 'net-net' charon[4392]: 06[CFG] loaded certificate "C=CH, O=Linux strongSwan, CN=moon.strongswan.org" from 'moonCert.pem' charon[4392]: 06[CFG] added configuration 'net-net' charon[4392]: 06[CFG] received stroke: initiate 'net-net' charon[4392]: 07[IKE] initiating IKE_SA net-net[1] to 10.10.99.2 charon[4392]: 07[IKE] initiating IKE_SA net-net[1] to 10.10.99.2 charon[4392]: 07[ENC] generating IKE_SA_INIT request 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) N(HASH_ALG) ] charon[4392]: 07[NET] sending packet: from 10.10.99.1[500] to 10.10.99.2[500] (1124 bytes) charon[4392]: 09[NET] received packet: from 10.10.99.2[500] to 10.10.99.1[500] (481 bytes) charon[4392]: 09[ENC] parsed IKE_SA_INIT response 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) CERTREQ N(HASH_ALG) N(MULT_AUTH) ] charon[4392]: 09[IKE] received cert request for "C=CH, O=Linux strongSwan, CN=strongSwan Root CA" charon[4392]: 09[IKE] sending cert request for "C=CH, O=Linux strongSwan, CN=strongSwan Root CA" charon[4392]: 09[IKE] authentication of 'moon.strongswan.org' (myself) with RSA_EMSA_PKCS1_SHA256 successful charon[4392]: 09[IKE] sending end entity cert "C=CH, O=Linux strongSwan, CN=moon.strongswan.org" charon[4392]: 09[IKE] establishing CHILD_SA net-net charon[4392]: 09[IKE] establishing CHILD_SA net-net charon[4392]: 09[ENC] generating IKE_AUTH request 1 [ IDi CERT N(INIT_CONTACT) CERTREQ IDr AUTH SA TSi TSr N(MULT_AUTH) N(EAP_ONLY) ] charon[4392]: 09[NET] sending packet: from 10.10.99.1[500] to 10.10.99.2[500] (1596 bytes) charon[4392]: 10[NET] received packet: from 10.10.99.2[500] to 10.10.99.1[500] (1532 bytes) charon[4392]: 10[ENC] parsed IKE_AUTH response 1 [ IDr CERT AUTH SA TSi TSr N(AUTH_LFT) ] charon[4392]: 10[IKE] received end entity cert "C=CH, O=Linux strongSwan, CN=sun.strongswan.org" charon[4392]: 10[CFG] using certificate "C=CH, O=Linux strongSwan, CN=sun.strongswan.org" charon[4392]: 10[CFG] using trusted ca certificate "C=CH, O=Linux strongSwan, CN=strongSwan Root CA" charon[4392]: 10[CFG] checking certificate status of "C=CH, O=Linux strongSwan, CN=sun.strongswan.org" charon[4392]: 10[CFG] fetching crl from 'http://crl.strongswan.org/strongswan.crl' ... charon[4392]: 10[LIB] unable to fetch from http://crl.strongswan.org/strongswan.crl, no capable fetcher found charon[4392]: 10[CFG] crl fetching failed charon[4392]: 10[CFG] certificate status is not available charon[4392]: 10[CFG] reached self-signed root ca with a path length of 0 charon[4392]: 10[IKE] authentication of 'sun.strongswan.org' with RSA_EMSA_PKCS1_SHA256 successful charon[4392]: 10[IKE] IKE_SA net-net[1] established between 10.10.99.1[moon.strongswan.org]...10.10.99.2[sun.strongswan.org] charon[4392]: 10[IKE] IKE_SA net-net[1] established between 10.10.99.1[moon.strongswan.org]...10.10.99.2[sun.strongswan.org] charon[4392]: 10[IKE] scheduling reauthentication in 3279s charon[4392]: 10[IKE] maximum IKE_SA lifetime 3459s charon[4392]: 10[IKE] CHILD_SA net-net{1} established with SPIs c70d5aba_i cb7e5654_o and TS 192.168.183.0/24 === 192.168.184.0/24 charon[4392]: 10[IKE] CHILD_SA net-net{1} established with SPIs c70d5aba_i cb7e5654_o and TS 192.168.183.0/24 === 192.168.184.0/24 vpn[4421]: + sun.strongswan.org 192.168.184.0/24 == 10.10.99.2 -- 10.10.99.1 == 192.168.183.0/24 charon[4392]: 10[IKE] received AUTH_LIFETIME of 3391s, scheduling reauthentication in 3211s > ipsec statusall Status of IKE charon daemon (strongSwan 5.3.5, Linux 4.4.0-93-generic, x86_64): uptime: 28 minutes, since Sep 15 18:34:16 2017 malloc: sbrk 2727936, mmap 0, used 616576, free 2111360 worker threads: 11 of 16 idle, 5/0/0/0 working, job queue: 0/0/0/0, scheduled: 3 loaded plugins: charon test-vectors aes rc2 sha1 sha2 md4 md5 random nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs8 pkcs12 pgp dnskey sshkey pem openssl fips-prf gmp agent xcbc hmac gcm attr kernel-netlink resolve socket-default connmark farp stroke updown eap-identity eap-sim eap-sim-pcsc eap-aka eap-aka-3gpp2 eap-simaka-pseudonym eap-simaka-reauth eap-md5 eap-gtc eap-mschapv2 eap-dynamic eap-radius eap-tls eap-ttls eap-peap eap-tnc xauth-generic xauth-eap xauth-pam xauth-noauth tnc-tnccs tnccs-20 tnccs-11 tnccs-dynamic dhcp lookip error-notify certexpire led addrblock unity Listening IP addresses: 10.10.133.101 10.10.99.1 192.168.183.250 Connections: net-net: 10.10.99.1...10.10.99.2 IKEv2 net-net: local: [moon.strongswan.org] uses public key authentication net-net: cert: "C=CH, O=Linux strongSwan, CN=moon.strongswan.org" net-net: remote: [sun.strongswan.org] uses public key authentication net-net: child: 192.168.183.0/24 === 192.168.184.0/24 TUNNEL Security Associations (1 up, 0 connecting): net-net[1]: ESTABLISHED 28 minutes ago, 10.10.99.1[moon.strongswan.org]...10.10.99.2[sun.strongswan.org] net-net[1]: IKEv2 SPIs: 98a6d3eeeaf1f9c0_i* a25f94aff1cdec12_r, public key reauthentication in 25 minutes net-net[1]: IKE proposal: AES_CBC_128/HMAC_SHA1_96/PRF_HMAC_SHA1/MODP_2048 net-net{2}: INSTALLED, TUNNEL, reqid 1, ESP SPIs: c0b03d25_i c07fc07c_o net-net{2}: AES_GCM_16_256/ESN, 94353752 bytes_i, 5965332882 bytes_o, rekeying in 11 seconds net-net{2}: 192.168.183.0/24 === 192.168.184.0/24 net-net{3}: INSTALLED, TUNNEL, reqid 1, ESP SPIs: c770da0f_i c8fe02d2_o net-net{3}: AES_GCM_16_256/ESN, 0 bytes_i, 0 bytes_o, rekeying in 11 minutes net-net{3}: 192.168.183.0/24 === 192.168.184.0/24 > ip xfrm state src 10.10.99.1 dst 10.10.99.2 proto esp spi 0xc07fc07c reqid 1 mode tunnel replay-window 0 flag af-unspec aead rfc4106(gcm(aes)) 0xcbcc5acbe14362c202a98286a72ea9b25f2f054435413c7097e476ed3e7c6d0b5d6c6f1d 128 anti-replay context: seq 0x0, oseq 0x3ef313, bitmap 0x00000000 src 10.10.99.2 dst 10.10.99.1 proto esp spi 0xc0b03d25 reqid 1 mode tunnel replay-window 0 flag af-unspec aead rfc4106(gcm(aes)) 0x58ae85c2c38b8102aabc3d8f14d62988e8935ebbc84a83eff13a3bd831a5fa1b36768b3b 128 anti-replay context: seq 0x0, oseq 0x0, bitmap 0x00000000 > cat /proc/crypto | grep -C10 pcrypt name : seqiv(rfc4106(gcm(aes))) driver : seqiv(pcrypt(rfc4106-gcm-aesni)) module : seqiv priority : 500 refcnt : 6 selftest : passed internal : no type : aead async : yes blocksize : 1 ivsize : 8 maxauthsize : 16 geniv : <none> name : rfc4106(gcm(aes)) driver : pcrypt(rfc4106-gcm-aesni) module : pcrypt priority : 500 refcnt : 6 selftest : passed internal : no type : aead async : yes blocksize : 1 ivsize : 8 maxauthsize : 16 geniv : <none> > cpuinfo (8 CPUs for each VM instance): processor : 7 vendor_id : GenuineIntel cpu family : 6 model : 79 model name : Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz stepping : 1 microcode : 0xb00001f cpu MHz : 2197.455 cache size : 25600 KB physical id : 14 siblings : 1 core id : 0 cpu cores : 1 apicid : 14 initial apicid : 14 fpu : yes fpu_exception : yes cpuid level : 20 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts nopl xtopology tsc_reliable nonstop_tsc aperfmperf eagerfpu pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch epb fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 invpcid rtm rdseed adx smap xsaveopt dtherm ida arat pln pts bugs : bogomips : 4394.91 clflush size : 64 cache_alignment : 64 address sizes : 42 bits physical, 48 bits virtual power management: > ip xfrm policy src 192.168.184.0/24 dst 192.168.183.0/24 dir fwd priority 2883 tmpl src 10.10.99.2 dst 10.10.99.1 proto esp reqid 1 mode tunnel src 192.168.184.0/24 dst 192.168.183.0/24 dir in priority 2883 tmpl src 10.10.99.2 dst 10.10.99.1 proto esp reqid 1 mode tunnel src 192.168.183.0/24 dst 192.168.184.0/24 dir out priority 2883 tmpl src 10.10.99.1 dst 10.10.99.2 proto esp reqid 1 mode tunnel src 0.0.0.0/0 dst 0.0.0.0/0 socket in priority 0 src 0.0.0.0/0 dst 0.0.0.0/0 socket out priority 0 src 0.0.0.0/0 dst 0.0.0.0/0 socket in priority 0 src 0.0.0.0/0 dst 0.0.0.0/0 socket out priority 0 src ::/0 dst ::/0 socket in priority 0 src ::/0 dst ::/0 socket out priority 0 src ::/0 dst ::/0 socket in priority 0 src ::/0 dst ::/0 socket out priority 0 -- Sven Anders <and...@anduras.de> () UTF-8 Ribbon Campaign /\ Support plain text e-mail ANDURAS intranet security AG Messestrasse 3 - 94036 Passau - Germany Web: www.anduras.de - Tel: +49 (0)851-4 90 50-0 - Fax: +49 (0)851-4 90 50-55 Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety. - Benjamin Franklin
<<attachment: anders.vcf>>