Hoi folks,
I run the lb plugin on a mix of amd64 servers and arm64 DPUs. On the
arm64 DPUs, I noticed a VIP occasionally sending connections to
application servers that aren't configured for that VIP. With a VIP that
had only one active AS, a small fraction of requests reached the
application servers of a busier VIP on the same load balancer. The amd64
load balancers never did this, and I discovered that arm64 uses the
non-SSE4.2 branch of lb_hash_get() in src/plugins/lb/lbhash.h
When it walks a sticky bucket, it returns the AS of the first live slot
that does not match the packet's hash and VIP, and skips a slot that
does match. The sticky table is per worker and shared by all VIPs. A
flow landing in a bucket whose first slot holds some other live flow
gets that flow's AS, whatever VIP it belongs to, for the whole connection.
I wrote a minimal repro for this in /test_lb_sticky_cross_vip. /This
test fails at HEAD on arm64 and passes with the accompanying fix in
lb_hash_get(). You can make the test fail on amd64 as well, by setting
LB_HASH_DO_NOT_USE_SSE_BUCKETS=1. The test shrinks the sticky table to
one bucket, sends one flow to VIP B and then three flows with two
packets each to VIP A. It then checks that every VIP A packet goes to
VIP A's AS. Without the fix the new test fails because a VIP A packet is
sent to VIP B's AS.
I attempted to make the scalar branch behave like the SSE4.2 version:
the first expired slot is noted as free, and a hit is only a live slot
with the same hash and VIP. See https://gerrit.fd.io/r/c/vpp/+/46795 and
I welcome review for it.
groet,
Pim
--
Pim van Pelt
PBVP1-RIPE -https://ipng.ch/
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#27182): https://lists.fd.io/g/vpp-dev/message/27182
Mute This Topic: https://lists.fd.io/mt/121260774/21656
Group Owner: [email protected]
Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/14379924/21656/631435203/xyzzy
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
