On 1/4/2019 8:10 PM, Akhil Goyal wrote: > > On 1/4/2019 5:59 PM, Ananyev, Konstantin wrote: >> Hi Akhil, >> >>> Hi Konstantin, >>> >>> With this patchset, I am seeing a 3% drop in legacy mode lookaside and >>> 12% drop with -l option. >>> I am debugging this. Will let you know if I find something. >> Ok, thanks. >> For legacy mode do you know which patch in the series causing 3% drop? >> Is it still: " fix crypto-op might never get dequeued" or a different one? >> Konstantin >> > for legacy mode you may consider this diff in 3/10 > > diff --git a/examples/ipsec-secgw/ipsec-secgw.c > b/examples/ipsec-secgw/ipsec-secgw.c > index 4f7a77d8d..d183862b8 100644 > --- a/examples/ipsec-secgw/ipsec-secgw.c > +++ b/examples/ipsec-secgw/ipsec-secgw.c > @@ -1015,10 +1015,13 @@ main_loop(__attribute__((unused)) void *dummy) > > if (nb_rx > 0) > process_pkts(qconf, pkts, nb_rx, portid); > - } > +// } > > - drain_inbound_crypto_queues(qconf, &qconf->inbound); > - drain_outbound_crypto_queues(qconf, &qconf->outbound); > + if (UNPROTECTED_PORT(portid)) > + drain_inbound_crypto_queues(qconf, > &qconf->inbound); > + else > + drain_outbound_crypto_queues(qconf, > &qconf->outbound); > + } > } > } > > The 3% gap was on single core performance. But on multi cores scenario, > there is no significant drop. > > But I see a bigger functional problem in case of non-legacy mode. > I am trying a multi tunnel scenario with 32 kind of flows on one side > each with different tunnels. > The flows are kind of dest ip: 192.168.101.1 ........... 192.168.x.1 > ......... 192.168.132.1. > Each IP has a different tunnel. > All are pumped from same port. I can see that some of the IPs are not > getting forwarded. > like 192.168.103.1, 192.168.104.1, 192.168.105.1, 192.168.114.1, > 192.168.115.1, 192.168.116.1, 192.168.122.1, 192.168.125.1, > 192.168.126.1, 192.168.130.1 > > The same setup/flows works perfectly fine with legacy mode.
I see that the IPs are not fixed and changes on every run. Probably, the inbound /outbound queues gets flushed when they do not intend to on that particular core and packets get discarded as they come from non-destined port. Also the issue does not come if I add the above suggested change. > > >>> -Akhil >>> >>> On 1/4/2019 1:55 AM, Konstantin Ananyev wrote: >>>> This patch series depends on the patch series: >>>> >>>> ipsec: new library for IPsec data-path processing >>>> http://patches.dpdk.org/patch/49410/ >>>> http://patches.dpdk.org/patch/49411/ >>>> http://patches.dpdk.org/patch/49412/ >>>> http://patches.dpdk.org/patch/49413/ >>>> http://patches.dpdk.org/patch/49414/ >>>> http://patches.dpdk.org/patch/49415/ >>>> http://patches.dpdk.org/patch/49416/ >>>> http://patches.dpdk.org/patch/49417/ >>>> http://patches.dpdk.org/patch/49418/ >>>> http://patches.dpdk.org/patch/49419/ >>>> >>>> to be applied first. >>>> >>>> v5 -> v6 >>>> Address issues reported by Akhil: >>>> segfault when using lookaside-proto device >>>> HW IPv4 cksum offload not enabled by default >>>> crypto-dev dequeue() is called to often >>>> >>>> v4 -> v5 >>>> - Address Akhil comments: >>>> documentation update >>>> spell checks spacing etc. >>>> introduce rxoffload/txoffload parameters >>>> single SA for ipv6 >>>> update Makefile >>>> >>>> v3 -> v4 >>>> - fix few issues with the test scripts >>>> - update docs >>>> >>>> v2 -> v3 >>>> - add IPv6 cases into test scripts >>>> - fixes for IPv6 support >>>> - fixes for inline-crypto support >>>> - some code restructuring >>>> >>>> v1 -> v2 >>>> - Several bug fixes >>>> >>>> That series contians few bug-fixes and changes to make ipsec-secgw >>>> to utilize librte_ipsec library: >>>> - changes in the related data structures. >>>> - changes in the initialization code. >>>> - changes in the data-path code. >>>> - new command-line parameters to enable librte_ipsec codepath >>>> and related features. >>>> - test scripts to help automate ipsec-secgw functional testing. >>>> >>>> Note that right now by default current (non-librte_ipsec) code-path >>>> will be used. User has to run application with new command-line option >>>> ('-l') >>>> to enable new codepath. >>>> The main reason for that: >>>> - current librte_ipsec doesn't support all ipsec algorithms >>>> and features that the app does. >>>> - allow users to run both versions in parallel for some time >>>> to figure out any functional or performance degradation with the >>>> new code. >>>> >>>> Test scripts were run with the following crypto devices: >>>> - aesni_mb >>>> - aesni_gcm >>>> - qat >>>> >>>> Konstantin Ananyev (10): >>>> examples/ipsec-secgw: allow user to disable some RX/TX offloads >>>> examples/ipsec-secgw: allow to specify neighbour mac address >>>> examples/ipsec-secgw: fix crypto-op might never get dequeued >>>> examples/ipsec-secgw: fix outbound codepath for single SA >>>> examples/ipsec-secgw: make local variables static >>>> examples/ipsec-secgw: fix inbound SA checking >>>> examples/ipsec-secgw: make app to use ipsec library >>>> examples/ipsec-secgw: make data-path to use ipsec library >>>> examples/ipsec-secgw: add scripts for functional test >>>> doc: update ipsec-secgw guide and relelase notes >>>> >>>> doc/guides/rel_notes/release_19_02.rst | 14 + >>>> doc/guides/sample_app_ug/ipsec_secgw.rst | 159 +++++- >>>> examples/ipsec-secgw/Makefile | 5 +- >>>> examples/ipsec-secgw/ipsec-secgw.c | 480 ++++++++++++++---- >>>> examples/ipsec-secgw/ipsec.c | 101 ++-- >>>> examples/ipsec-secgw/ipsec.h | 67 +++ >>>> examples/ipsec-secgw/ipsec_process.c | 357 +++++++++++++ >>>> examples/ipsec-secgw/meson.build | 6 +- >>>> examples/ipsec-secgw/parser.c | 91 ++++ >>>> examples/ipsec-secgw/parser.h | 8 +- >>>> examples/ipsec-secgw/sa.c | 263 +++++++++- >>>> examples/ipsec-secgw/sp4.c | 35 +- >>>> examples/ipsec-secgw/sp6.c | 35 +- >>>> examples/ipsec-secgw/test/common_defs.sh | 153 ++++++ >>>> examples/ipsec-secgw/test/data_rxtx.sh | 62 +++ >>>> examples/ipsec-secgw/test/linux_test4.sh | 63 +++ >>>> examples/ipsec-secgw/test/linux_test6.sh | 64 +++ >>>> examples/ipsec-secgw/test/run_test.sh | 80 +++ >>>> .../test/trs_aescbc_sha1_common_defs.sh | 69 +++ >>>> .../ipsec-secgw/test/trs_aescbc_sha1_defs.sh | 67 +++ >>>> .../test/trs_aescbc_sha1_esn_atom_defs.sh | 5 + >>>> .../test/trs_aescbc_sha1_esn_defs.sh | 66 +++ >>>> .../test/trs_aescbc_sha1_old_defs.sh | 5 + >>>> .../test/trs_aesgcm_common_defs.sh | 60 +++ >>>> examples/ipsec-secgw/test/trs_aesgcm_defs.sh | 66 +++ >>>> .../test/trs_aesgcm_esn_atom_defs.sh | 5 + >>>> .../ipsec-secgw/test/trs_aesgcm_esn_defs.sh | 66 +++ >>>> .../ipsec-secgw/test/trs_aesgcm_old_defs.sh | 5 + >>>> .../test/tun_aescbc_sha1_common_defs.sh | 68 +++ >>>> .../ipsec-secgw/test/tun_aescbc_sha1_defs.sh | 70 +++ >>>> .../test/tun_aescbc_sha1_esn_atom_defs.sh | 5 + >>>> .../test/tun_aescbc_sha1_esn_defs.sh | 70 +++ >>>> .../test/tun_aescbc_sha1_old_defs.sh | 5 + >>>> .../test/tun_aesgcm_common_defs.sh | 60 +++ >>>> examples/ipsec-secgw/test/tun_aesgcm_defs.sh | 70 +++ >>>> .../test/tun_aesgcm_esn_atom_defs.sh | 5 + >>>> .../ipsec-secgw/test/tun_aesgcm_esn_defs.sh | 70 +++ >>>> .../ipsec-secgw/test/tun_aesgcm_old_defs.sh | 5 + >>>> 38 files changed, 2727 insertions(+), 158 deletions(-) >>>> create mode 100644 examples/ipsec-secgw/ipsec_process.c >>>> create mode 100644 examples/ipsec-secgw/test/common_defs.sh >>>> create mode 100644 examples/ipsec-secgw/test/data_rxtx.sh >>>> create mode 100644 examples/ipsec-secgw/test/linux_test4.sh >>>> create mode 100644 examples/ipsec-secgw/test/linux_test6.sh >>>> create mode 100644 examples/ipsec-secgw/test/run_test.sh >>>> create mode 100644 >>>> examples/ipsec-secgw/test/trs_aescbc_sha1_common_defs.sh >>>> create mode 100644 examples/ipsec-secgw/test/trs_aescbc_sha1_defs.sh >>>> create mode 100644 >>>> examples/ipsec-secgw/test/trs_aescbc_sha1_esn_atom_defs.sh >>>> create mode 100644 >>>> examples/ipsec-secgw/test/trs_aescbc_sha1_esn_defs.sh >>>> create mode 100644 >>>> examples/ipsec-secgw/test/trs_aescbc_sha1_old_defs.sh >>>> create mode 100644 examples/ipsec-secgw/test/trs_aesgcm_common_defs.sh >>>> create mode 100644 examples/ipsec-secgw/test/trs_aesgcm_defs.sh >>>> create mode 100644 >>>> examples/ipsec-secgw/test/trs_aesgcm_esn_atom_defs.sh >>>> create mode 100644 examples/ipsec-secgw/test/trs_aesgcm_esn_defs.sh >>>> create mode 100644 examples/ipsec-secgw/test/trs_aesgcm_old_defs.sh >>>> create mode 100644 >>>> examples/ipsec-secgw/test/tun_aescbc_sha1_common_defs.sh >>>> create mode 100644 examples/ipsec-secgw/test/tun_aescbc_sha1_defs.sh >>>> create mode 100644 >>>> examples/ipsec-secgw/test/tun_aescbc_sha1_esn_atom_defs.sh >>>> create mode 100644 >>>> examples/ipsec-secgw/test/tun_aescbc_sha1_esn_defs.sh >>>> create mode 100644 >>>> examples/ipsec-secgw/test/tun_aescbc_sha1_old_defs.sh >>>> create mode 100644 examples/ipsec-secgw/test/tun_aesgcm_common_defs.sh >>>> create mode 100644 examples/ipsec-secgw/test/tun_aesgcm_defs.sh >>>> create mode 100644 >>>> examples/ipsec-secgw/test/tun_aesgcm_esn_atom_defs.sh >>>> create mode 100644 examples/ipsec-secgw/test/tun_aesgcm_esn_defs.sh >>>> create mode 100644 examples/ipsec-secgw/test/tun_aesgcm_old_defs.sh >>>>
