I can’t tell you what it would take to modify the code. I can observe that RFC 8926 section 3.3 explicitly states that the source port should be stable for a given flow, and should be varied across flows. It specifically suggests that “the source port SHOULD be calculated using a hash of the encapsulated packet headers using, for example, a traditional 5-tuple.”
Yours, Joel From: vpp-dev@lists.fd.io <vpp-dev@lists.fd.io> On Behalf Of Pim van Pelt via lists.fd.io Sent: Thursday, January 13, 2022 5:37 PM To: vpp-dev <vpp-dev@lists.fd.io> Subject: [vpp-dev] VXLAN and RSS Hoi folks, I did a deep dive today on VXLAN, GENEVE and compared it to GRE and L2XC - the full read is here: https://ipng.ch/s/articles/2022/01/13/vpp-l2.html<https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-454445555731-eb58b434aa85fefe&q=1&e=da8dd7f1-6228-4705-822d-acdf096c6cbb&u=https%3A%2F%2Fipng.ch%2Fs%2Farticles%2F2022%2F01%2F13%2Fvpp-l2.html> One thing that I observed is that both VXLAN and GENEVE use static source ports. In the case of VLLs, (an l2 xconnect from a customer ethernet interface into a tunnel), the customer port will be receiving IPv4 or IPv6 traffic (either tagged or untagged) and this allows the NIC to use RSS to assign this inbound traffic to multiple queues, and thus multiple CPU threads. That’s great, it means linear encapsulation performance. However,, once the traffic is encapsulated, it’ll become single flow with respect to the remote host, ie we're sending from 10.0.0.0:4789<https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-454445555731-3b084ba08ac5cdbf&q=1&e=da8dd7f1-6228-4705-822d-acdf096c6cbb&u=http%3A%2F%2F10.0.0.0%3A4789%2F> to the remote 10.0.0.1:4789<https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-454445555731-92908955b444a8b7&q=1&e=da8dd7f1-6228-4705-822d-acdf096c6cbb&u=http%3A%2F%2F10.0.0.1%3A4789%2F> and it is for this reason, that all decapsulation is single threaded. One common approach is to use an ingress hash algorithm to choose from a pool of source ports, or possibly a simpler round-robin over a pool of ports 4000-5000, say, based on the inner payload. That way, the remote would be able to use multiple RSS queues. However, VPP currently does not implement that. I think the original author has this in mind as a future improvement based on the comment on L295 in vxlan.c /* UDP header, randomize src port on something, maybe? */ udp->src_port = clib_host_to_net_u16 (t->src_port); udp->dst_port = clib_host_to_net_u16 (t->dst_port); What would it take for src_port to not be static? It would greatly improve VXLAN (and similarly, GENEVE) throughput on ingress. groet, Pim -- Pim van Pelt <p...@ipng.nl<mailto:p...@ipng.nl>> PBVP1-RIPE - http://www.ipng.nl/<https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-454445555731-370b7382a99e002e&q=1&e=da8dd7f1-6228-4705-822d-acdf096c6cbb&u=http%3A%2F%2Fwww.ipng.nl%2F>
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#20709): https://lists.fd.io/g/vpp-dev/message/20709 Mute This Topic: https://lists.fd.io/mt/88408739/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
