Hello,
I looked at the patchset (v9) and I found no obvious problems, but I miss some instrumentation to understand what is going on. For example, how many flows are offloaded, or how many per second, etc... We can definitely work on that as a follow up. I have a MLX5 (16.20.1010) which is connected to a traffic generator. The results are unexpected and I don't know why yet. I will continue on it next week. The flow is pretty simple, just echo the packet back: ovs-ofctl add-flow ovsbr0 in_port=10,action=output:in_port This is the result without HW offloading enabled: Partial: 14619675.00 pps 7485273570.00 bps Partial: 14652351.00 pps 7502003940.00 bps Partial: 14655548.00 pps 7503640570.00 bps Partial: 14679556.00 pps 7515932630.00 bps Partial: 14681188.00 pps 7516768670.00 bps Partial: 14597427.00 pps 7473882390.00 bps Partial: 14712617.00 pps 7532860090.00 bps pmd thread numa_id 0 core_id 2: packets received: 53859055 packet recirculations: 0 avg. datapath passes per packet: 1.00 emc hits: 53859055 megaflow hits: 0 avg. subtable lookups per megaflow hit: 0.00 miss with success upcall: 0 miss with failed upcall: 0 avg. packets per output batch: 28.20 idle cycles: 0 (0.00%) processing cycles: 12499399115 (100.00%) avg cycles per packet: 232.08 (12499399115/53859055) avg processing cycles per packet: 232.08 (12499399115/53859055) Based on the stats, it seems 14.7Mpps is the maximum a core can do it. This is the result with HW Offloading enabled: Partial: 10713500.00 pps 5485312330.00 bps Partial: 10672185.00 pps 5464158240.00 bps Partial: 10747756.00 pps 5502850960.00 bps Partial: 10713404.00 pps 5485267400.00 bps pmd thread numa_id 0 core_id 2: packets received: 25902718 packet recirculations: 0 avg. datapath passes per packet: 1.00 emc hits: 25902697 megaflow hits: 0 avg. subtable lookups per megaflow hit: 0.00 miss with success upcall: 0 miss with failed upcall: 0 avg. packets per output batch: 28.11 idle cycles: 0 (0.00%) processing cycles: 12138284463 (100.00%) avg cycles per packet: 468.61 (12138284463/25902718) avg processing cycles per packet: 468.61 (12138284463/25902718) 2018-05-18T22:34:57.865Z|00001|dpif_netdev(dp_netdev_flow_8)|WARN|Mark id for ufid caaf720e-5dfe-4879-adb9-155bd92f9b 40 was not found 2018-05-18T22:35:02.920Z|00002|dpif_netdev(dp_netdev_flow_8)|WARN|Mark id for ufid c75ae6c5-1d14-40ce-b4c7-6d5001a458 4c was not found 2018-05-18T22:35:05.160Z|00105|memory|INFO|109700 kB peak resident set size after 10.4 seconds 2018-05-18T22:35:05.160Z|00106|memory|INFO|handlers:1 ports:3 revalidators:1 rules:5 udpif keys:2 2018-05-18T22:35:21.910Z|00003|dpif_netdev(dp_netdev_flow_8)|WARN|Mark id for ufid 73bdddc9-b12f-4007-9f12-b66b4bc189 3e was not found 2018-05-18T22:35:21.924Z|00004|netdev_dpdk(dp_netdev_flow_8)|ERR|rte flow creat error: 2 : message : flow rule creati on failure Looks like offloading didn't work and now the tput rate is lower, which is not expected either. I plan to continue on this and I appreciate if you have any idea of what is going on. Thanks, fbl _______________________________________________ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev